By Hilary J. Schmidt, Ph.D., Co-Founder, Caliber Institute for Quality Medical Education
Editor’s Note: This article provides key insights from a 2016 Alliance Meeting Session led by Damon Marquis and Hilary J. Schmidt.
Why is there a need for an effective instructional design process that can accelerate behavior change and performance improvement? At various sessions of the Alliance 2016 Annual Meeting sessions, three areas of concern were highlighted: 1) gaps in healthcare; 2) gaps in CE design; and 3) the concomitant need for more effective CE. Enhancing education design so that it can drive measurable improvements in healthcare is part of the solution.
Gaps in healthcare reveal that we have a long way to go.
- It takes, on average, 17 years for new science and guidelines to become pervasively integrated into practice.
- S. patients receive only 50 percent of the recommended care (Health for Life, AHA Association, 2007).
- The U.S. has the highest medical error rates and worst level of coordination of care in comparison to other major industrialized countries (How the U.S. Healthcare System Compares Internationally, Commonwealth Fund, 2014).
Gaps in CE design show we can be more effective.
- Research shows that CE activities can drive behavior change to improve care, but less than 20 percent actually does, and fewer than 5 percent leads to measurable improvements in patient outcomes.
- ACCME data reveals:
- 98 percent of certified activities are designed to change competence.
- 61 percent of activities are designed to change performance, but only 42 percent actually measures performance changes.
- Only 32 percent of activities are designed to improve patient outcomes, and only 13 percent of these actually measure changes in patient outcomes.
Calls to evolve to more effective CE shows we have direction.
- ACCME’s chair, Dr. Graham McMahn, called on providers to continue to innovate to make CE activities “more relevant, efficient, effective, memorable and customized.” He made the point that if you miss out on any one of these attributes, the CE activity will fail to drive the intended behavior changes needed to improve patient care.
- Funders called on providers to optimize the design of education so that it more reliably leads to measurable improvements in patient care; they noted that a common pitfall is the lack of alignment between the gap, learning objectives, the target audience, the instructional methods and outcomes measures. They also noted the critical importance of differentiating between gaps that require targeting of the heathcare team (or system) as opposed to (or in addition to) the individual.
The Effective Use of the Instructional Design Process Can Help Close These CE Educational Gaps
Instructional design is a systematic process that applies principles of cognition and behavior change to create education experiences that make learning (of individuals or teams) more efficient, more effective, more enduring and more appealing. An effective ID process uses tested theories of teaching and learning to analyze the underlying cognitive and behavioral reasons for gaps in practice (i.e., root cause analysis) and then designs instructional formats and methods that are best able to address these root cause(s). Formats that lead to rapid and enduring learning vary considerably depending on the type of cognitive or behavioral changes that are required to close the gap (i.e., to address the root cause). Evaluation is designed to measure changes in the targeted knowledge, skills and/or attitudes, and the extent to which the gap has been closed. The ability to meet a number of the ACCME criteria can be facilitated and enhanced through the application of a comprehensive instructional design process. (See C2, C3, C5, C16, C18, C19, C11). Instructional design is a profession and many universities and colleges provide both undergraduate and graduate degrees in instructional design.
What are the Steps in Instructional Design Process?
The six essential steps in an effective design process are summarized below. These steps should be followed in sequence, as each step has an impact on the next step.
- Define the healthcare gap: A healthcare gap is a gap between expected patient outcomes and actual patient outcomes as defined by quality measures, data and literature. Healthcare gaps can be traced to a wide variety of issues such as errors in screening, diagnosing, managing or treating patients, system flaws or a lack of patient adherence. Representative examples of healthcare gaps include: too many unnecessary fractures due to undiagnosed or untreated osteoporosis; too many unnecessary complications from diabetes due to failure to achieve adequate glycemic control; or excessive, avoidable DVTs post-surgery due to failure to prophylax. Closing a defined healthcare gap is the ultimate goal.
- Identify the root causes for the gap: This is a critical — but frequently overlooked — step. Understanding the origins, or root causes, for a targeted gap is critical in designing education that can close that gap. Different root causes require different types of interventions. Not all root causes can best be addressed via education. Most CE (probably greater than 80 percent) assumes that the root cause for a healthcare gap is a lack of individual HCPs’ knowledge or know-how (i.e., competence). However, root cause analysis reveals that initial assumptions about the root cause are almost never correct. A single healthcare gap may involve multiple root causes, and the root cause for the same gap may be different in different individuals, teams or settings. For example, consider possible reasons for why osteoporosis is frequently not diagnosed or treated. Perhaps the cost is too high? Screening centers are not easily accessible? HCPs don’t know the treatment guidelines? HCPs understand the guidelines but don’t follow them? Or patients are diagnosed but don’t follow through on treatment? The appropriate educational content, target audience(s) and instructional format(s) will be determined by which of these possibilities is, in fact, the root cause(s).
A number of years ago, a group of endocrinologists undertook a careful exploration of the root cause for this gap in osteoporosis screening. This revealed that bone density testing centers were accessible, women followed through on testing when ordered and cost (at that time) was not a factor. It was determined that HCPs were failing to order bone density tests not due to a lack a knowledge about the guidelines but because they didn’t know how to read and interpret bone density tests. When the root cause was re-conceptualized as a skills gap — as opposed to a knowledge gap — education shifted from focusing on the knowledge ofosteoporosis (e.g., burden, pathophysiology, prevalence, incidence, screening guidelines, Tx options, etc.) to developing skills in reading and interpreting bone density tests. (Most quality groups in healthcare systems or organizations are experienced in utilizing rigorous root cause analysis methods.)
- Define the target audience(s): Exposing the root cause will define the target audience(s) and their needs. If a lack of HCP knowledge or skill is identified, then the individuals responsible for performing a task should be the target. In contrast, if the root cause involves the system, improving coordination of care, team’s work, transitions of care, etc., then the teams or groups responsible become the target as opposed to individuals.
- Write the learning objectives so they align with the root cause: Begin with the end in mind. Learning objectives should be action-oriented and should state the intended end goal practice behaviors that will result from participating in the educational activity; they state behaviors directly aligned with the root cause that are currently missing but required to close the healthcare gap. In the osteoporosis example, the end goal, expected after the education, would be that HCPs would be able to “read and interpret bone density tests.” This “begin with the end in mind” approach makes it easier to focus content, design appropriate methods and define outcomes measures.
- Develop an outcomes plan: Don’t wait to develop outcomes measures until the activity has been fully developed. The learning objectives, if well-aligned with the root cause, will make it very clear what measures make the most sense. In the osteoporosis example, outcomes measures included pre-post-assessments of: 1) confidence in the ability to read and interpret bone density tests; 2) proficiency in reading bone density tests; and 3) actual changes in the probability of partcipants’ ordering a bone density test (via chart reviews).
- Design the instructional methods to target the root cause: At this stage in the process, cognitive and behavioral science principles are applied to design content, methods and formats that: 1) engage the audience (create a need to learn at the outset via a predisposing activity); 2) accelerate understanding of new concepts; 3) transfer learning from short-term into long-term memory; and 4) increase the recall and application of new knowledge and skills at the point of care. This is where principles of adult learning play a role. If teams or systems are targeted, additional principles from behavioral economics, systems theories, etc., may be relevant as well. This process makes CE “relevant, efficient, effective and memorable.” In the osteoporosis example, instruction shifted from didactic presentations that pushed out knowledge to interactive workshops in which participants received practice and feedback in reading and interpreting bone density tests for a broad range of cases — until a level of proficiency was developed. The results of this shift in education led to a 12-fold increase in the number of bone density tests ordered for participants in the activity and significant improvements in HCP confidence and proficiency in reading bone density tests. Incorporating practice and feedback is a critical design element for developing skills that is often neglected, and many healthcare gaps require that HCPs, or teams, replace old habits with new skills.
In sum, a systematic instructional design process is an effective way to address the ACCME criteria and to design education that conforms to Moore’s established framework for planning and assessing educational activities. Most importantly, it provides a proven framework to effectively design education that can lead to behavior changes that drive improvements in healthcare quality.
To learn more about the instructional design process, you may find the following resource helpful.
Dirksen, J. Design for How People Learn, New Riders, Berkley CA, 2012