Abstract Background Large Language Models (LLMs) offer a novel means of translating complex medical information into patient-friendly language. Patients with complex perianal fistulising Crohn’s disease (pfCD) indicated their preference for AI-generated patient-friendly summaries with actionable recommendations in a recent global survey on attitudes to imaging in pfCD (1). This study evaluated the clinical utility, safety, and patient acceptability of GPT-4o in generating AI-assisted summaries of MRI fistula reports. Methods A three-phase mixed-methods study was conducted at a single centre. Phase I involved prompt engineering and pilot testing of GPT-4o outputs for feasibility and clinical plausibility. Phase II assessed consecutive MRI fistula scans over a two-month period, with reports reviewed by radiologists, gastroenterologists, and surgeons to determine accuracy, hallucination rates, readability, and thematic content. Phase III randomised patients to review either a simple or complex fistula case, each containing an original report and an AI-generated summary (order randomised, origin blinded), and rate readability, trustworthiness, usefulness and comprehension. Results Sixteen patients participated in Phase I pilot testing. In Phase II, 250 consecutive scans were assessed by clinicians. In Phase III, 61 patients evaluated paired original and AI-generated summaries (Figure 1). Across both patient phases, AI summaries scored significantly higher for readability, comprehension, and usefulness than original reports (all p 0.001), with equivalent trust ratings (Table 1). Mean Flesch-Kincaid scores were markedly higher for AI summaries (66 vs. 26; p 0.001). Clinicians highlighted improved anatomical structuring and accessible language, but noted hallucinations (11 %), anatomical mislabelling, omissions, and unverified recommendations. A revised template incorporating MDT-focused action points and a lay summary section was co-developed. Conclusion LLMs can enhance the readability and patient understanding of complex MRI reports but remain limited by factual inaccuracies and inconsistent terminology. Safe adoption requires structured oversight, domain-specific fine-tuning, and clinician validation. Future development should focus on standardised structured reporting templates incorporating clinician-approved lay summaries, co-designed with patients, and validated across radiology and other medical specialties. Reference: (1) Anand E, Devi J, Antoniou A, Joshi S, Stoker J, Lung P, et al. Patient’s attitudes to Magnetic resonance imaging in perianal fistulising Crohn’s disease: a global survey. Crohn’s otaf015. https://doi.org/10.1093/crocol/otaf015. Conflict of interest: Mr. Anand, Easan: No conflict of interest Ghersin, Itai: No conflict of interest Lingam, Gita: No conflict of interest Devlin, Katie: No conflict of interest Pelly, Theo: No conflict of interest Singer, Daniel: Author Daniel Singer was employed by the company Tenrec Analytics. Tomlinson, Chris: No conflict of interest Munro, Robin EJ: Author Robin E. J. Munro reports ownership of stock in the company IQVIA and is an employee of IQVIA Capstick, Rachel: No conflict of interest Antoniou, Anna: Author Anna Antoniou was employed by the company Novartis UK Hart, Ailsa: Grant: Takeda Personal Fees: Abbvie, Amgen, Arena, AZ, Falk, Celltrion, Eli Lilly, Ferring, Genentech/ Roche, GSK, Pfizer, Takeda, Napp, Pharmacosmos, Janssen (J & J), Bristol-Myers Squibb, Gilead, Galapagos Sahnan, Kapil: Author Kapil Sahnan has received honoraria for symposia from the companies Intuitive, Johnson & Johnson and Medtronic. Tozer, Philip: Personal Fees: Takeda - speaker, member of Inspire, and advisory boards Ferring - speakers fees Falk - speakers fees Tillott’s - speakers fees J&J - speakers fees Lung, Phillip FC: No conflict of interest Figure 1: Study Flowchart Table 1: Patient Evaluation of Original vs AI Report summarie
Anand et al. (Thu,) studied this question.