AI driven Hybrid CNN transformer model for early detection and severity assessment of diabetic retinopathy

Abstract As the major cause of sight impairment in working age adults is diabetic retinopathy (DR) which requires timely and proper diagnosis to avoid the irreversible vision loss. Traditional retinal fundus image based manual diagnosis is time consuming, subjective, and requires highly skilled ophthalmologists, and therefore is not scalable in resource constrained environments. Automated DR grading systems are essential to overcome this issue especially in underserved areas. Although Convolutional Neural Networks (CNNs) are suitable in capturing the fine grained local features namely Microaneurysms and exudates, they are incapable of considering the global retinal context, which is required to accurately grade the severity. Transformer models, on the other hand, are more effective in capturing long range dependencies, at the cost of low spatial resolution. To address these shortcomings, this research suggests Fusion Net a hybrid DL model that combines a CNN branch that extracts local features in the form of GoogLeNet with a Vision Transformer (ViT) branch that captures global context. This is a dual stream architecture that combines complementary embeddings to categorize DR into five ordinal levels of severity. The model has been trained on a mixed dataset of APTOS 2019 and Messidor-2 and tested on three datasets that are not used during training. Fusion Net had overall accuracy percent of 98.85, AUC-ROC of 0.981 and weighted F1-score percent of 97.62. It was also highly sensitive in detecting early stage DR and balanced in all the severity levels. The thorough ablation experiments, ordinal assessment scales, and feature map confirms the strength and clarity of the suggested framework. The findings indicate that Fusion Net is an effective, explainable, and computationally efficient system to use in grading automated DR with a high chance of implementation into clinical practice and tele-ophthalmology systems.