Journal of CME (07/21/23) Lucero, Katie Stringer; Moore, Donald E.
Assessing continuing medical education (CME) outcomes on statistical significance alone overlooks effect size, researchers assert. Medscape's Katie Stringer Lucero and Vanderbilt University School of Medicine's Donald E. Moore write that pre- and post-assessment questions should aspire to promote active learning; evaluate the educational intervention's effects according to learning objectives; probe learners' progress in behavior change; and gain better insights into what educational elements work, who benefits and why. "Questions or quizzes can facilitate self-assessment and awareness of what one knows and where one needs more education," they write. "They are used in continuing education as a method for creating active learning — or participation from the learner in the learning process." The authors note that data which is not independent may counter assumptions of statistical tests employed for pooled data analysis, suggest that exclusive concentration on statistical significance may mislead evaluators, and point out that statistical significance is only one facet of CME outcome analyses. In terms of behavioral change assessment, "[w]e suspect different measured outcome profiles depending on the stage of change in which a learner is," Lucero and Moore write. Pooled data may thus miss key information that must be factored in if CME is to advance best-in-class patient care. This demands "an understanding of whether the learner had primarily a reinforcing or learning something new experience; that needs to be overlaid with self-efficacy, intent to change and current practice to get a full picture of understanding the process of behavior change," the authors advise. Given the heterogeneity in most learner groups, they also suggest analyzing pre- and post-assessment data with other factors to understand moderators and mediators from descriptive segmentation and multivariate evaluations. The authors they aim "to shine a light on an important issue (evaluation beyond outcomes) that needs to be addressed so the field of CME/CPD can move forward with quality studies that will contribute to improved clinician performance and patient health."
Read More