Development of a Survey Instrument Grounded in Resident Voice to Measure Key Drivers of Learning in Simulation

Simulation based education (SBE) is frequently driven by competencies necessary to track progress in residency. What is missing are crucial factors that matter to the residents themselves. To date, there is no validated curricular tool that captures experiences grounded in factors the residents themselves say matter. The objective of this study was to uncover resident derived factors that meaningfully shape learning in SBE to inform development of a learner centered tool.

This was a qualitative study held at an academic center with an accredited emergency medicine (EM) residency. Focus groups were conducted among residents in all classes. Investigators performed an iterative thematic analysis through independent and comparative coding. Additions and changes were cataloged, and themes were developed and refined. An additional investigator settled conflicts and provided confirmations. Member checking processes were conducted. Theme definitions were decided upon, and exemplar quotes were used in alignment of theme generation for survey development.

4 major themes emerged, each serving as a distinct section in the survey. Each section was paired with a descriptive anchor to describe the theme and its key components. The Psychological Safety and Learning Effectiveness theme encompasses how residents engage meaningfully in a learning zone that balances being challenged with feeling safe to take risks. The Integration of Experience and Evidence theme connects SBE to real, evidence informed practices of EM. The Learner Responsive Simulation Design theme captures aspects residents identify as integral to learning, emphasizing alignment between the degree of challenge and the resident’s developmental stage and metacognitive maturity. The Clinical Reasoning Structure theme provides an understanding for how residents translate simulation practice to clinical practice. 19 questions were developed across the 4 sections.

We conclude that a survey instrument grounded in resident voice and responsive to dynamic changes across residency classes was developed and is positioned to measure key drivers of learning in SBE. This tool will next undergo a validation process in preparation for distribution. This tool may inform simulation curricula while amplifying resident voice, guide targeted program improvements, and enable large scale institutional comparison.