Beyond International Classification of Diseases Codes: Automated Out-of-Hospital Cardiac Arrest Case Identification From History and Physical Notes Using a PubMedBERT Classifier

Accurate case identification of out-of-hospital cardiac arrest (OHCA) in electronic health records (EHR) remains challenging. Administrative diagnosis codes frequently misclassify cases, often capturing in-hospital arrests, transfers, and non-arrest conditions. As a result, case identification commonly relies on manual review of clinical documentation, a resource-intensive process that limits scalability and sustainability of national registries.

We developed a transformer-based deep learning model (OSCAR: Out-of-hospital Sudden Cardiac Arrest Recognition) to identify OHCA from the History of the Present Illness (HPI) section of intensive care unit History and Physical admission notes (ICU H&Ps). The model integrates a pretrained language model (PubMedBERT) with engineered contextual features capturing note structure, arrest locations, timing, negation, and transfer status, which are jointly learned through a multilayer classification head. The model was trained on hospital discharge summaries using manually annotated OHCA labels, focal loss to address class imbalance and validated on a manually annotated dataset validated on external dataset of ICU H&Ps from the University of Chicago. Performance was evaluated across operating thresholds, with emphasis on clinically meaningful, high-specificity operating points.

In external validation on ICU H&Ps, the PubMedBERT-based model demonstrated strong discriminative performance. At a reporting threshold of 0.94, sensitivity was 81.8%, specificity 90.3%, positive predictive value 0.794, negative predictive value 0.916, and F1 score 0.806. Performance remained stable across nearby thresholds.

A PubMedBERT-based NLP classifier applied to ICU H&Ps identified OHCA cases with high specificity and strong overall performance in external validation. These results support the feasibility of accurate automated text-based approaches for OHCA case identification and offer a scalable alternative to manual chart review.