A multimodal multi-instance learning model using echocardiography and health records predicted peak VO2 with an R² of 0.603 and identified high-risk heart failure patients with an AUROC of 0.849.
Observational (n=1,127)
Yes
Does a multimodal multi-instance learning model using TTE and EHR data improve the prediction of peak VO₂ and identification of high-risk heart failure patients compared to prior single-instance models?
A multimodal multi-instance AI framework using routine echocardiography and EHR data can accurately predict CPET-derived peak VO₂ and identify high-risk heart failure patients, potentially expanding access to advanced risk stratification.
Absolute Event Rate: 0.849% vs 0.836%
Heart failure (HF) is a progressive and fatal disease that affects nearly 7 million individuals in the United States, with prevalence expected to surpass 10 million by 2040. Cardiopulmonary exercise testing (CPET) represents the gold standard for assessing functional capacity and predicting survival outcomes among HF patients but its widespread use is limited by practical constraints. Here we introduce a multimodal multi-instance learning framework that predicts peak oxygen consumption (peak VO₂), a critical indicator from CPET, using the more accessible transthoracic echocardiography (TTE) studies and electronic health records (EHR). By modeling the cross-modal interactions and the multi-instance structure of TTE studies, our approach significantly improves predictive accuracy and generalization. The model achieves an R² of 0.603 in peak VO₂ prediction and AUROC of 0.849 in high-risk patient identification, surpassing prior work (R² = 0.529, AUROC = 0.836). On the external validation cohort, the model achieves an R² of 0.541 compared to 0.395 and an AUROC of 0.870 compared to 0.797 from previous work. The improved performance more accurately allows for identification of patients who may benefit from advanced heart failure therapies that otherwise may have been missed.
Huang et al. (2026) conducted an observational in Heart failure (n=1,127). Multimodal multi-instance learning AI model (TTE and EHR) vs. Prior single-instance ensemble AI model was evaluated on High-risk patient identification (AUROC) and peak VO2 prediction (R²). A multimodal multi-instance learning model using echocardiography and health records predicted peak VO2 with an R² of 0.603 and identified high-risk heart failure patients with an AUROC of 0.849.