Vitiligo state assessment based on progressive transfer learning and multimodal domain adaptation

Vitiligo is a common skin depigmentation disorder; assessing its state is crucial for the treatment outcome. Collecting multimodal data for vitiligo assessment is complex and costly in clinical practices, and the limited data size restricts the performance of deep learning models. Transfer learning can alleviate the shortage of training data in medical image recognition, however, its applications in vitiligo state assessment are constrained by feature differences between natural and medical images and insufficient generalization to different modalities. To address these challenges, this paper introduces a vitiligo state assessment method based on progressive transfer learning and multimodal domain adaptation. The scheme uses a large set of unlabeled medical images as a bridge to reduce the discrepancies between natural and medical images through multi-step fine-tuning. An adaptive parameter unfreezing strategy is then applied for the accurately labeled target data to improve the adaptability and accuracy of the model. In addition, an integrated multimodal domain adaptation approach based on CycleGAN and hue, saturation, value (HSV) color space transformation is proposed to reduce the impact of modality differences on transfer learning. Experimental results demonstrate that compared to the conventional transfer learning method, the proposed method improves the accuracy by 2.5% and 3.2% on the clinical and Wood's lamp modality datasets, respectively. This accurate vitiligo state assessment method can be applied to a wide range of multimodal dermatological diseases where labeled data is limited.