Immersive interactive intelligent patient educational system for venous thromboembolism

Abstract Background Large language models (LLMs) can serve as virtual teaching assistants, providing patients with detailed and relevant information and perhaps eventually interactive simulations. Aims This study explored the integration of LLMs into the education and patient-centered care for venous thromboembolism (VTE). Methods We developed a immersive interactive intelligent patient educational system for patients with VTE using LLM，Qwen1.5-7B, Text-to-Speech(TTS)，and Lip Synchronization(Lip-Sync) technologies. Results The immersive interactive intelligent patient educational system (ChatVTE) consisted of four components (Figure 1): 1) Information Collection of Multimodal Optical Character Recognition Technology Based on Large Models, which collecting patient's clinical data. 2) Knowledge, Attitude/Belief, Practice and Health Belief Model, which capturing patient's health requirement. 3) VTE relevant data were systematically gathered from open-access Internet sources and indexed into a knowledge database. We leveraged Retrieval-Augmented Language Modeling to recall this information and utilized it for pretraining, which was then integrated into Qwen1.5-7B, creating an VTE-specific knowledge question & answer platform. 4) Based on intelligent question & answer, a digital virtual doctor created with TTS Lip-Sync technologies to facilities patient education （Figure 2）. ChatVTE generated fewer hallucinations and demonstrated greater consistency，improved the quality of doctor-patient interaction and enhance the effectiveness of knowledge dissemination. Conclusion To the best of our knowledge, the digital virtual VTE doctor of ChatVTE was the first digital virtual doctor driven by specialty-specific VTE knowledge retrieval AI that utilizes the latest LLM. It appeared the promising for patient education and shared clinical decision support. Continued research and evaluation are necessary to ensure the optimal integration of AI-based interactive tools into patient-centered care.