Derivation and Validation of a Point-of-Care Ultrasound Assessment Tool for Focused Cardiac Ultrasound

No point of care ultrasound (POCUS) competency assessment tool has gained widespread acceptance among training programs. There is a lack of high-quality studies among diverse learners on how to measure POCUS competency. We set out to derive and internally validate a POCUS competency-based assessment tool that can be used across a diverse group of trainees by determining what elements of an assessment are most predictive of a high entrustment scale and independent performance.

This was a prospective cohort study on a convenience sample of medical students, residents, fellows and faculty with varying POCUS knowledge and experience. Participants filled out a pre-survey with questions about prior experience with POCUS. The assessment tool was piloted and refined with the final tool consisting of 30 elements covering knowledge, skill, clinical integration and an entrustment scale. For each assessment 2 POCUS experts independently completed an assessment form, after they filled out one additional adjudicated assessment. Participants with an adjudicated entrustment score of ≥4 was used as the gold standard to determine competency. A multivariable logistic regression model was selected using LASSO with 10-fold cross validation, under the restriction that included at least one each of “knowledge” variables and “image acquisition” variables. Interrater reliability for individual binary components and entrustment was performed with Kappa.

Assessments were completed on 76 (57%) students, 48 (36%) residents, 8 (6%) attendings, and 2 (1%) fellows. Twenty-seven (20%) participants had an adjudicated entrustment ≥4. Of the 30 elements assessed, the final prediction model consisted of 6 elements: total number of prior POCUS exams performed, ability to identify anatomy on a parasternal long view, obtain a subxiphoid view with adequate depth, identify a poor left ventricular ejection fraction on a POCUS video, name a treatment for cardiogenic shock, and identify a cause of right ventricular dilation. This model predicted an entrustment ≥4 with a cross-validated area under the curve of 0.98. Weighted kappa for entrustment was 0.93 (0.89-0.97).

Six elements were predictive of a high entrustment scale and independent performance of cardiac POCUS, with excellent interval validity. Next steps will aim to evaluate this tool using an external multi-center cohort of trainees.