1138: Using Natural Language Processing to Identify Advance Care Planning in Pediatric Critical Illness

Introduction: Large multisite trials evaluating advance care planning (ACP) interventions require scalable, reliable methods to identify ACP discussions in the medical record. Manual chart review, the current standard, is labor-intensive and variable. Natural language processing (NLP) may offer a feasible alternative, but its use in pediatric critical illness has not been validated. This study evaluates an NLP approach to detect ACP documentation compared to manual chart review. Methods: We conducted a retrospective cohort study of 128 children under 21 years of age admitted to a large U.S. pediatric intensive care unit following out-of-hospital cardiac arrest between 2013 and 2024. Patients were identified using diagnosis codes for cardiopulmonary arrest, with additional inclusion criteria of non-neonatal ICU admission and hospital stay over 24 hours. Clinical and demographic data were abstracted manually, including pediatric palliative care (PPC) consultation and ACP documentation. A rules-based NLP algorithm was developed and applied to all clinical notes to detect three ACP domains: goals of care conversations, limits to life-sustaining therapies, and hospice discussions. NLP results were compared with manual review using standard test statistics. Results: Among 128 patients, 26% identified as racial or ethnic minorities, 33% were female, 42% had a prior complex chronic condition, and 18% received PPC consultation. Iterative refinement of the keyword library yielded robust performance statistics across ACP domains (F1 score = 1). ACP documentation was found in 60% of patients with a higher prevalence among those receiving PPC consultation (100%). Goals of care conversations were most frequently documented (60%), followed by limitations to life-sustaining therapies (41%) and hospice (1.6%). NLP identified ACP documentation with 99% specificity, 100% sensitivity, and significantly reduced review time (1 week vs 6 months). Conclusions: A rules-based NLP approach can feasibly and efficiently identify ACP documentation in pediatric critical illness with accuracy comparable to manual abstraction. This method may support scalable measurement of ACP in future multisite trials of communication interventions.