Accurate classification of breast cancer subtypes is critical for personalized treatment planning and prognostic assessment. While histopathology and RNA-seq data each provide complementary insights into tumor biology, their integration remains challenging due to heterogeneity in data modalities. This study proposes a robust multimodal framework that combines histology-based features extracted using the CTransPath vision transformer with transcriptomic profiles to improve breast cancer subtype classification and interpretability. We developed an end-to-end pipeline incorporating three learning paradigms: RNA-Only, WSI-Only, and multimodal fusion. Histological features were extracted using a pretrained CTransPath encoder at 20× magnification (0.5 μm/pixel) with non-overlapping 256×256-pixel patches, while RNA-seq features were standardized and projected into a shared latent space. We evaluated multiple fusion techniques including gated attention, cross-attention, and concatenation. To enhance transparency, attention rollout was applied to visualize feature importance across whole-slide images. An uncertainty-aware smart routing mechanism was introduced to selectively use RNA-Only predictions for high-confidence cases or defer to multimodal inference for ambiguous samples, optimizing computational efficiency. Evaluated on the TCGA-BRCA dataset comprising 924 patients with PAM50 molecular subtypes, our routing-based multimodal model achieved 94.93% accuracy, outperforming both unimodal baselines and fixed fusion strategies. The routing mechanism required multimodal inference for only 38.2% of samples, achieving a 3.12× computational speedup while maintaining superior accuracy. Confusion matrices revealed significant improvements in distinguishing closely related subtypes such as Luminal A and Luminal B. Attention rollout provided interpretable heatmaps localizing discriminative histological regions associated with each subtype, aligning with established pathological criteria. This study demonstrates that integrating CTransPath-derived histology features with transcriptomic profiles through confidence-aware routing offers a practical, explainable, and computationally efficient approach for breast cancer subtype classification suitable for clinical decision support systems.
Nabil et al. (Wed,) studied this question.