Reliability and Validity of a Follow-up Severity Index for Tests Pending at Discharge

Test results pending at discharge (TPAD) from the emergency department (ED) pose a patient safety risk, yet EDs often lack systematic approaches to manage high TPAD volumes. To address this gap, we developed the Follow-Up Severity Index (FUSI), a 5-point ordinal scale mirroring the emergency severity index (1 = most urgent, 5 = least urgent). We describe the reliability and validity of TPAD triage using FUSI.

We conducted a retrospective observational study using encounters with TPADs from 12 EDs and 4 urgent cares from a single health system. We performed a purposeful stratified sampling within the 20 most common TPAD types (e.g., urine culture, viral swabs) to ensure case diversity (n = 100 total). Standardized vignettes were written by a study physician from EHR review. Two emergency physicians independently assigned FUSI scores using a prespecified flowchart. An expert physician blinded to FUSI assigned the ideal follow-up for each vignette from prespecified options (call the patient to refer to ED, call the patient to modify prescriptions, call the patient to re-evaluate, forward the result to the primary care provider, and take no further action) mapped to an ordinal scale. Primary outcomes were interrater reliability, measured with Cohen κ, and validity, measured by the correlation between FUSI and the follow-up action. The association between FUSI and whether the follow-up required a patient phone call was assessed with logistic regression.

Interrater reliability for FUSI had weighted κ 0.91 (95% CI: 0.80-0.99). FUSI scores were strongly associated with follow-up action (ρ = 0.76, p<0.001), with greater triage urgency indicating more intensive actions. Each 1-point increase in FUSI urgency was associated with 5.38-fold higher odds of requiring a patient call ( 3.06-9.43).

FUSI showed excellent interrater reliability and strong validity against expert-defined follow-up actions. More urgent FUSI scores align with more intensive patient contact, suggesting FUSI may support the prioritization of ED TPADs.