What type of study is this?

This is a Validation Study study.

What question did this study set out to answer?

The research focuses on developing an advanced computational framework for classifying histological grades of invasive ductal carcinoma (IDC).

March 14, 2026Open Access

A novel multimodal framework integrating pathomics, deep learning, and machine learning for breast cancer histological grades classification

Key Points

The research focuses on developing an advanced computational framework for classifying histological grades of invasive ductal carcinoma (IDC).
Developed a multimodal framework for automated grading of IDC from H&E whole-slide images.
Assembled a multi-center dataset with 3,660 images from 925 IDC patients.
Extracted multi-modality features using deep learning models and standardized radiomics descriptors.
Utilized an Attention MIL framework for feature fusion and comparison of fusion strategies.
Achieved an internal accuracy of 92.5% and external accuracy of 90.9%.
Area under the curve (AUC) values were 93.5% (internal) and 92.1% (external).
Demonstrated that deep features significantly enhance classification accuracy when combined.

Abstract

The present study is aimed at developing and testing a strong, interpretable, and multi-scale computational pathology framework for the automated histologic grade (grade-based) grading of invasive ductal carcinoma (IDC) from hematoxylin and eosin (H&E) whole-slide images (WSIs). A multi-center dataset consisting of 3,660 WSIs from 925 patients with IDC from five institutions was assembled and quality controls were very stringent. Images were captured at four magnifications (4X, 10X, 20X, 40X), in order to capture both a view of global tissue architecture and fine detail of the nuclei. Multi-modality features were extracted: low and high magnification deep features (DINOv2 ViT-S/16, ConvNeXt V2 Small), 71 handcrafted radiomics descriptors extracted from H&E channels and three pseudo-nuclei indices. Correlation filtering and four feature selection approaches (mutual information, recursive feature elimination, ANOVA, LASSO) were used on a modality per modality basis. An Attention MIL framework with a novel cross scale attention module and cross scale consistency loss was used for feature fusion and grading. Three strategies of fusion (early, late, hybrid) were compared. Model performance was evaluated using 5-folder cross-validation, internal, and external validation on an independent cohort (n = 922 WSIs). The hybrid fusion of LASSO selected features resulted in the best performance as the accuracy was found to be 92.5% (internal) and 90.9% (external) and the AUC were found to be 93.5% and 92.1% respectively. High magnitude deep features proved to be the most informative single modality and multi-modality fusion increased both the accuracy and robustness. The proposed framework was better than end-to-end baselines (p < 0.001), and the recall was balanced across all grades. Attention heatmaps identified regions of diagnostic interest which were consistent with pathologist annotations. The multi-scaled, multi modal Attention-MIL framework provides accurate, interpretable and generalizable grading of IDC, outperforming conventional and end-to-end methods. Its inter-institutional strength and biological interpretability make it an appealing candidate for a clinical decision-support tool.

A novel multimodal framework integrating pathomics, deep learning, and machine learning for breast cancer histological grades classification

Key Points

Abstract

Cite This Study