Generative artificial intelligence for patient drug information: a new framework for content construction

Abstract Introduction Simplifying medication information, using plain language, and improving layout structure can enhance patient comprehension.1 Generative artificial intelligence (Gen AI) has growing applications in pharmaceutical care, yet the quality and efficiency of its output varies with the chosen models and prompting strategies.2 Aim To compare multiple large language models (LLMs) and prompting strategies for generating patient drug information and determine an optimal Gen-AI–based workflow. Methods Three medications—pregabalin capsules, acetaminophen extended-release tablets, and levofloxacin tablets—were selected. Eight LLMs (ChatGPT-4o, DeepSeek-R1, Grok 3, and Gemini 2.5 Flash, Doubao, Doubao Deep Thinking, Kimi, Kimi Long Thinking k1.5,) with five prompt strategies (Zero-Shot, Zero-Chain-of-Thought Zero-CoT, Tree-of-Thought ToT, Zero-Shot accessibility was scored by the number of human-AI interactions (maximum 3). Quality was defined as the mean total score across dimensions of nine outputs per combination, and stability as the standard deviation (SD). Inter-rater reliability was assessed using a two-way random-effects intraclass correlation coefficient (ICC). Discrepancies were resolved through discussion. The optimal combination was identified through integrated analysis of quality and stability. Results A total of 360 outputs were generated. Inter-rater reliability was excellent (ICC = 0.93; 95% CI: 0.88–0.97; p 0.001). There are significant differences in the quality and stability. Quality scores ranged from 19.16 (Grok 3 + ZC&FS) to 26.15 (DeepSeek-R1 + ZS&FS). Stability (SD) ranged from 0.31 (Gemini 2.5 Flash + ZC&FS) to 7.42 (ChatGPT + Zero-Shot). DeepSeek-R1 consistently ranked among the highest-performing models, and its combination with ZS&FS was identified as the optimal workflow when balanced between quality and stability. Rankings are shown in Fig. 1. Conclusion This study provides a structured, multidimensional evaluation of LLMs and prompting strategies and has identified an optimal framework for generating patient drug information using generative AI. With pharmacist review, AI-generated materials can support accurate, readable, and accessible medication information, promoting rational drug use. Limitations include the lack of patient usability testing and the evolving nature of LLM capabilities.