Abstract Rationale Pulmonary embolism (PE) is life-threatening but easy to miss. CT pulmonary angiogram (CTPA) scanning is a key method to diagnose PE, but it has risks. There are concerns CTPA scanning is over-utilised. We used a large language model (LLM) to review seven years of CTPA scans at Royal Berkshire Hospital (RBH), Reading, UK, to investigate this. Methods To analyse trends in CTPA scanning, we obtained all CTPA scan reports performed between 1st June 2017 and 2nd May 2025, along with linked metadata including demographics and D-dimer results. We used the pre-trained Qwen 3-8B LLM within a secure research environment to analyse anonymised CTPA reports. Guided by structured prompts, the model automatically extracted diagnostic information including PE presence, anatomical location, size, heart strain, and other relevant radiological features. Model performance was validated against manual extraction by two independent physicians in a randomly selected subset of 200 scans. Using LLM-derived outputs, we evaluated scan utilization and positivity. Spearman’s rank correlation and logistic regression were applied to assess trends in quarterly scan numbers and proportions of positive scans Results 15674 scans were performed at RBH between 1st June 2017 and 2nd May 2025. Scan numbers increased over time (1,531 in 2018 vs. 2,273 in 2024; p 0.001) (Figure). Comparing against the 200 manually reviewed reports, the LLM correctly labelled all positive cases of PE (36/36, sensitivity100%), however misclassified some negative reports (155/164, specificity 94%), yielding a positive predictive value of 80%. Overall, the LLM labelled 3536/15674 (22.5%) scans as positive. The LLM was unable to extract information on PE location, size or evidence of right heart strain from 751/3536 (21%) reports. Among the remaining 1336 were labelled as being segmental or greater and 889 reported evidence of right heart strain. The likelihood of scans showing low volume PE (i.e. small or subsegmental PE) was relatively constant across the data (Figure, logistic regression model estimate= -0.0056, 95% confidence interval (-0.035,0.024)) Proportion positive was higher pre-covid (2017-2019, N = 953/4127(23%)) compared to post-covid (2022-2025, N = 1539/7135 (21%)) (p = 0.011). Conclusions CTPA scan numbers at RBH have increased over the past seven years, as has the total number of PEs detected. This increase is not attributable to greater detection of small PEs. Notably however, contrary to similar previous studies, the proportion of positive scans has fallen slightly in recent years, suggesting some CTPA scans may be unnecessary. This abstract is funded by: National Institues of Health and Care Research, UK
Davies et al. (2026) studied this question.