Currently, the diagnosis of anterior cruciate ligament (ACL) diseases relies primarily on single-modality image, with most studies limited to classification. The paper proposes a novel framework called an ACL diagnosis and treatment decision support framework (ACL-DTF), which is the first to explore the integration of multimodal information with a large language model (LLM) in ACL research for disease diagnosis and preliminary treatment decision support. First, a novel segmentation network, termed UKSnet, is proposed to achieve high-quality ACL region segmentation (Dice=94.5%). Second, a multi-class image feature fusion algorithm (MIFF) is employed to extract the image modality features of the ACL (AUC=0.969). Third, an ACL image-text fusion module (AITF) is introduced, incorporating channel attention and multi-head cross-attention mechanisms to fuse extracted image and text features for ACL disease diagnosis (AUC=0.971). Finally, a two-step fine-tuning strategy for ChatGLM3, leveraging both public and private datasets, is designed to support personalized ACL treatment decision-making. Experimental results demonstrate that MIFF consistently outperforms existing comparative methods on one private dataset and two public datasets. Compared with the image-only baseline using our proposed UKSnet+MIFF Method, AITF improves accuracy by 1.3%, recall by 0.8%, precision by 2.5%, and F1 score by 1.8%. The safety, rationality, and fluency of the generated preliminary treatment decision supports were validated through quantitative, qualitative, and subjective evaluations. The proposed ACL-DTF improves diagnostic performance for ACL disease diagnosis and provides patient-specific preliminary decision support, highlighting the potential of multimodal–LLM integration in ACL-related clinical settings.
Yang et al. (Sun,) studied this question.