Virtual treatment for radiotherapy: Multimodal Generative Prediction of Longitudinal NSCLC Tumor Progression

Predicting the longitudinal evolution of tumors during radiotherapy is a complex and clinically critical challenge in medical imaging analysis, especially when models are conditioned solely on imaging data. This work introduces a Virtual Treatment (VT) framework that formulates non-small cell lung cancer (NSCLC) tumor progression as a multimodal conditional image-to-image translation paradigm, where the generative process is driven not only by past CT scans but also by treatment-related and patient-specific clinical information. Using a private longitudinal dataset of 222 NSCLC patients with 895 CT scans acquired during radiotherapy, it is shown that incorporating diverse modalities such as delivered dose increments, demographic characteristics, histological diagnosis, and tumor staging (cT, cN) improves the ability of generative models to predict plausible future anatomical states. A benchmarking study is conducted across four families of generative models including 2D GANs, 2.5D diffusion models, and fully 3D latent diffusion architectures, and showing that multimodal conditioning is essential for capturing patient-specific radiobiological dynamics. Unlike traditional image-based synthesis methods, the proposed VT framework leverages these complementary modalities to refine the generative trajectory, enabling the model to forecast future CT scans based on both anatomical evolution and clinical context, thereby reflecting realistic treatment-response patterns. Across quantitative metrics, tumor volumetric evaluation, and statistical tests, diffusion-based models conditioned on the combination of demographic and tumor-related features achieved the most stable and dose-aware tumor evolution forecasts. These findings underscore the importance of multimodal information for generative modeling in longitudinal oncology and position VT as a promising tool for in-silico treatment monitoring and adaptive radiotherapy support in NSCLC. • A multimodal generative Virtual Treatment framework predicts dose-dependent CT evolution in NSCLC. • A comprehensive benchmark contrasts 2D GANs with 2.5D/3D diffusion models, revealing paradigm-specific strengths and limitations. • Multimodal clinical conditioning and tumor-focused loss markedly improve tumor-response fidelity. • Diffusion models outperform GANs in stability, anatomical fidelity, and longitudinal realism.