Association Between Rosh Review Question Volume and In-Training Examination Performance for Emergency Medicine Residents

Emergency medicine (EM) residency training emphasizes mastery of didactic knowledge to ensure success on the American Board of Emergency Medicine (ABEM) Qualifying Examination. The In-Training Examination (ITE) serves as a validated surrogate of board preparedness, yet limited data exist regarding the impact of commercial question banks, particularly Rosh Review, on performance across postgraduate years (PGY). We sought to evaluate the association between the number of Rosh Review questions completed during residency and subsequent performance on mock ITEs and the national ITE.

We conducted a retrospective cohort study of EM residents at an urban teaching program from 2021–2024. Data included annual ITE and mock ITE scores, number of Rosh Review questions completed, and raw question bank accuracy. Bivariate analyses included descriptive statistics and pairwise correlations by PGY level. Mixed-effects linear regression, clustered by resident, was used for multivariate modeling of year-to-year score change, controlling for sex, age, PGY, and prior test score (including squared terms to account for ceiling effects).

Seventy-one residents (32% female, mean age 29 years) contributed 174 ITE and 163 mock ITE observations. Mean ITE score was 77.3 (SD 8.9), mean mock ITE score 71.5 (SD 7.2), and mean total questions completed 1,537 (SD 955). In bivariate analysis, ITE scores correlated with mock ITE scores (r=0.73, p<0.001) and with total question volume (r=0.26, p<0.01). In multivariate regression, every additional 100 Rosh Review questions was associated with a 0.17-point increase in ITE score change (β=0.168, 95% CI 0.068–0.269, p=0.001). No independent association was observed between question volume and mock ITE performance.

Greater completion of Rosh Review questions was independently associated with statistically significant but modest improvements in ITE performance. The magnitude of effect suggests that, while question bank use may contribute to incremental gains, it is unlikely to be the sole driver of board preparedness. Structured QBank use should be considered as one adjunct among multiple educational strategies. Larger, multi-institutional studies are warranted to better define clinically meaningful thresholds for question volume and to clarify downstream impact on ABEM Qualifying Examination success.