Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction

Abstract

Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram-based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury. Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer (MM-DiT) framework that jointly estimates RP risk and characterizes the sources of predictive uncertainty. MM-DiT integrates planning computed tomography (CT) images and three-dimensional radiation dose distributions through self-supervised multimodal pre-training, reducing reliance on limited and potentially noisy toxicity labels. The resulting representations are further refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation. A learnable label-noise model is incorporated to explicitly account for uncertainty arising from imperfect toxicity annotations. Therefore, it provides individualized RP risk estimates with complementary measures of aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability at the individual-patient level. We evaluated MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty. The results demonstrate its potential to provide accurate RP risk estimates while quantifying clinically relevant sources of predictive uncertainty.