TBDD : Three‐Branch Disease‐Driven Model for Medical Report Generation

ABSTRACT The automatic generation of radiology reports aims to produce accurate descriptions of patients' radiographic images, thereby effectively reducing the workload of radiologists. Although recent research has made notable progress, aligning visual features with report semantics and generating long‐form reports remain key challenges. To address these issues, we propose a Three‐Branch Disease‐Driven model (TBDD). Specifically: For the image‐report retrieval branch, we build upon traditional contrastive learning methods by leveraging the disease‐tagged characteristics of X‐ray images to extract shared disease‐level information from both images and reports. This facilitates semantic querying of visual features, providing diagnostic prior knowledge to the encoder and enhancing the semantic richness of the decoder. A disease diagnosis network is constructed to extract diagnostic features as guiding information, further improving the accuracy of report generation. Given that the descriptions of the same disease tend to be similar across reports, this network helps reinforce the precision of generated content. A multibranch feature fusion module is designed to map critical features from different branches into the main network, enabling multisource information to contribute effectively to report generation. Experimental results on two major radiology report datasets, IU‐Xray and MIMIC‐CXR, demonstrate that the proposed method outperforms existing state‐of‐the‐art approaches in both language generation metrics and clinical accuracy.