AEM Education and Training (07/22/22) Pokrajac, Nicholas; Roszczynialski, Kelly N.; Rider, Ashley; et al.

Researchers developed and validated an Objective Structured Supervision Evaluation (OSSE) for assessing faculty supervision skills during medical procedures. The authors modeled the OSSE after the Objective Structured Teaching Evaluation (OSTE), and selected endotracheal intubation as a proof of concept. Instrument development included literature review, adaptation of an appropriate published OSTE exercise, selection of 10 medical education and airway management experts for content review, a panel-conducted Delphi process to agree on instrument items, and testing for proof of validity incorporating Kane's framework. Volunteer emergency medicine faculty residents at an academic medical facility qualified for the program. Each participated in a 20-minute scripted OSSE modeled after an actual clinical scenario, with their performance scored by three trained "standardized residents." The final instrument included 19 items incorporating three global rating measures. Testing domains included supervision climate, participant control of patient care, trainee assessment, instructional skills, case-specific measures, and general supervisor rating. Seventeen of 24 faculty participants met the minimum passing standard (MPS) initially, and all satisfied the MPS following simulation-based mastery learning. Average participant scores rose 19.5%, and faculty members had a higher median OSSE score than resident volunteers. The authors concluded that the OSSE shows promise, although "further testing is required to demonstrate improvement in patient-centered outcomes."

Read More