What does this research mean for the field?

The proposed multimodal deep learning framework achieves 99.06% accuracy in classifying diabetic foot ulcers by integrating synthetic thermal information with RGB images. Novelty: ClaimNovelty.NOVEL_FINDING. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

Aim to develop a robust deep learning framework that integrates RGB and synthetic thermal images for accurate DFU classification.

February 19, 2026Applied Sciences0 citationsOpen Access

View Full Paper

A Multimodal Dual-Stream Cross-Attention Deep Learning Framework for Diabetic Foot Ulcer Classification

MSMehmet Umut Salur

Key Points

Aim to develop a robust deep learning framework that integrates RGB and synthetic thermal images for accurate DFU classification.
Developed RGB2T-cGAN to generate synthetic thermal images from RGB data.
Created the Multimodal Dual-Stream Multi-Head Cross-Attention classifier for processing dual image streams.
Performed 10-fold GroupKFold cross-validation for validating model generalization across patient groups.
Achieved 99.06% accuracy and 99.09% recall on test datasets.
Outperformed single-modality models with 91.51% accuracy for RGB and 96.23% for thermal.
Demonstrated high generalization capability with average accuracy of 96.49 ± 1.04 and AUC value of 0.9927 ± 0.0067.

Abstract

Finding diabetic foot ulcers (DFUs) early and accurately is essential for improving patients’ quality of life and lowering the risk of amputation. RGB images, commonly used in automated DFU detection, have limitations such as lighting variations, color inconsistencies, and inability to directly reflect physiological information. Background/Objectives: Although thermal images can capture temperature anomalies associated with inflammation and circulatory disorders, they cannot provide consistent performance due to their low spatial resolution and limited availability in clinical datasets. Furthermore, the lack of paired RGB–thermal image pairs makes it difficult to develop effective multimodal deep learning models. Methods: This study proposes a two-stage multimodal deep learning approach to overcome these limitations. In the first stage, an RGB2T-cGAN (RGB to Thermal cGAN) model based on pix2pix was designed to generate synthetic thermal representations from RGB images that resemble clinical patterns, thereby addressing the missing modality problem. In the second stage, the Multimodal Dual-Stream Multi-Head Cross-Attention (MDS-MHCA) classifier model was developed, which processes DFU RGB and generated synthetic thermal images through separate streams, enabling the dynamic modeling of complementary information across modalities. Results: The proposed MDS-MHCA model achieved 99.06% accuracy, 99.09% recall, and 99.06% F1-score on the test set, demonstrating a clear advantage over models based solely on RGB (91.51% accuracy) or thermal (96.23% accuracy) modalities. Furthermore, patient-based 10-fold GroupKFold cross-validation results demonstrate that the model offers high generalization capability across different patient groups, with an average accuracy of 96.49 ± 1.04 and an AUC value of 0.9927 ± 0.0067. Conclusions: The findings reveal that the proposed approach, through the integration of synthetic thermal information and cross-attention-based multimodal fusion, overcomes the fundamental limitations of single-modality-based systems and offers a DFU detection system that is more robust and reliable and holds potential for integration into clinical decision support systems.

Demander à l'IA

Bookmark

View Full Paper

Demander à l'IA

Bookmark

View Full Paper

A Multimodal Dual-Stream Cross-Attention Deep Learning Framework for Diabetic Foot Ulcer Classification

Key Points

Abstract

Cite This Study