Abstract Background Deep learning has shown promise in diabetes management but faces challenges in real-world application due to its “black-box” nature, characterized by opaque internal decision-making processes. Explainable artificial intelligence (XAI) methods have been proposed to enhance model transparency. However, most of current XAI methods applied in the medical field often ignore the interaction of features in complex environments and pose deviation from clinical domain knowledge. Methods Our study used two Electronic Health Record (EHR) cohorts of hospitalized patients with type 2 diabetes (T2DM), including an internal dataset of 1,275 inpatients (mean age 58.5 ± 14.3 years) and an external dataset of 292 patients (mean age 69.3 ± 14.5 years). We introduce an expert-guided XAI framework to improve the transparency and trustworthiness of deep learning models for insulin titration in diabetes management. The framework utilizes a post-hoc XAI model named Shapley Taylor Interaction Index (STII) to capture the impact of feature interactions. Additionally, the model is refined iteratively in a doctor-in-the-loop (DIL) process by encoding clinical constraints to align with medical expertise. Results Here we show that our STII-DIL model could explore the interaction factors and reduce unreasonable explanations compared with other explanation models. The final XAI system explanations demonstrated strong alignment with experts’ explanations and increased correctness by expert evaluation An AI-human collaboration study revealed that insulin titration accuracy significantly improved for junior clinicians with STII-DIL assistance, while senior clinicians showed minimal change. Both junior and senior clinicians reported increased confidence when using the STII-DIL system. Conclusions We present an explainable deep learning framework that combines post-hoc XAI and expert domain knowledge to provide transparent and expert-aligned explanations for insulin titration in type 2 diabetes management. This framework enhances decision-making accuracy and confidence, especially for junior clinicians, and may facilitate broader clinical adoption of AI-assisted decision-making tools.
He et al. (Thu,) studied this question.