Abstract Background Many health systems rely on structured electronic health record (EHR) data to evaluate adherence to low-density lipoprotein cholesterol (LDL-C) management guidelines¹ in patients with atherosclerotic cardiovascular disease (ASCVD). However, quality assessments based solely on structured EHR data fail to capture potential explanations for guideline-discordant care, such as external laboratory results or patient drug intolerance. Purpose To assess (1) whether Generative Pretrained Transformer 4 omni (GPT-4o), a large language model (LLM), could accurately extract pertinent data from free-text (unstructured) clinical notes and (2) the extent to which incorporation of these data would affect quality measures of lipid management. Methods We used structured EHR data to analyze LDL-C levels and statin use of patients with ASCVD seen in cardiology clinics at a large academic medical center from 2022-2024. We then applied GPT-4o with zero-shot, chain-of-thought prompting to free-text cardiology clinic notes to (1) identify external LDL-C values documented within the notes of patients without a recent in-system lipid panel and (2) infer reasons for statin non-use among those with LDL-C ≥70 mg/dL. We validated GPT-4o's accuracy through manual review of 500 randomly selected charts. Results Among 9,098 patients with ASCVD (median age 71.0 years, 61.6% male, 70.5% White), structured EHR data showed 7,122 (78.3%) had recent lipid panels, of whom 3,456 (48.5%) had LDL-C 70 mg/dL (Figure 1). Compared to manual chart review, GPT-4o demonstrated high precision in capturing external LDL-C values (0.97) and reasons for statin non-use, including intolerance (0.91), physician decision (0.96), patient refusal (0.90), or not documented (0.95). Among 1,976 patients without recent in-system lipid panels, GPT-4o identified 359 (18.2%) with external lipids documented in clinical notes, of whom 171 (47.6%) had LDL-C 70 mg/dL. Incorporating these external results led to a higher lipid testing rate (82.2% vs. 78.3%), though the proportion achieving LDL-C 70 mg/dL remained unchanged (48.5%; Figure 2). Among 3,854 patients with LDL-C ≥70 mg/dL, 2,895 (75.1%) were on a statin and 1,755 (45.5%) were on a high-intensity statin (HIS). GPT-4o identified 582 patients with documented statin intolerance. Removing these patients from the analysis improved overall rates of statin (81.9% vs. 75.1%) and HIS use (53.6% vs. 45.5%). Conclusion GPT-4o can retrieve and interpret clinically relevant data from free-text notes with high precision. While integrating this unstructured data yielded only modest changes in quality measures of lipid management, it revealed the clinical contexts behind apparent care gaps. LLM-augmented quality assessment may guide targeted quality improvement initiatives for patients with ASCVD by identifying specific barriers to optimal lipid management, including statin intolerance, patient reluctance, and clinical inertia.Figure 1 Figure 2
Nguyen et al. (Sat,) studied this question.