Abstract Introduction The advent of large language models (LLMs) powered by artificial intelligence (AI) has opened up new avenues for patients to interpret their medical reports. Nevertheless, the clinical reliability and utility of these LLMs in healthcare settings remain to be established. Objectives This study aimed to assess the medical reliability of ChatGPT-4o and its capacity to generate patient-friendly reports based on coronary CT angiography (CCTA) findings. Methods We conducted a retrospective analysis using anonymised CCTA reports without any associated metadata, which were processed by ChatGPT-4o. A single-shot prompting approach was used via the OpenAI API key. We provided the following instructions to ChatGPT-4o: 1) "Summarise the report." 2) "Make it easy for the patient to understand." 3) "What do you suggest for this patient?" Two physicians independently assessed the quality and professional accuracy of both the original and AI-generated reports using a five-point Likert scale. Word counts and linguistic precision were recorded, while two laypersons evaluated the comprehensibility of the outputs. Statistical analysis involved Kappa and t-tests. Results A total of 300 CCTA reports were analysed from three Hungarian centres, encompassing various abnormal findings in line with their consecutive incidence (5% CABG, 5% stent, 5% non-diagnostic; 20% severe stenosis, 65% minimal-to-moderate stenosis). Inter-reviewer agreement was excellent. The overall reliability scores for the original reports (4.87±0.39), Prompt 1 (4.62±0.76), Prompt 2 (4.54±0.93), and Prompt 3 did not differ significantly (p0.005). Linguistic accuracy for Prompt 2 was notably high (4.92±0.28). Word counts were reduced by 49%, 45%, and 63% for Prompts 1–3, respectively, with no indications of AI-generated hallucinations. Performance was lower in graft-related cases for Prompts 1 and 2 (2.8±1.25, 2.8±1.25). Lay reviewers rated the patient-friendly Prompt 2 significantly more understandable than the original clinical reports (4.89±0.34 vs. 3.76±0.72, p0.005). Conclusions ChatGPT-4o can generate concise, accurate, and patient-friendly CCTA report summaries without significant professional errors or misleading claims. Although it performed less reliably for complex graft-related reports, its ability to enhance patient comprehension is promising and warrants further clinical evaluation.
Baksa et al. (Thu,) studied this question.