Beyond AUC: Do Mammography Risk Models Predict When Cancer Occurs?
Svetlana Krasnova, Andrey Krylov, Dmitry Sorokin
Abstract
Longitudinal deep learning models for mammography-based breast cancer risk prediction output either per-year cumulative risks or a discrete distribution over the year of diagnosis, from which cumulative risks F1,...,F5 are derived. These models are evaluated almost exclusively with year-wise AUCs and the concordance index. We show that these metrics are largely insensitive to when cancer occurs: they can be high even for a model that carries no information about the year of diagnosis. We re-evaluate two recent longitudinal models (OA-BreaCR, LMV-Net) on CSAW-CC with a 5-year horizon using timing metrics computed on cancer cases only. Despite C-indices up to 0.75, neither orders cancer cases by time to diagnosis significantly better than chance (ctime 0.51 and 0.55, both 95% CIs covering 0.5). Weighted absolute timing error favours a constant "year 3" prediction over both models, so we use ctime as the primary timing metric. For Mirai's additive-hazard head, also used by LMV-Net, we analyse how per-year increments passed through a ReLU can collapse under severe class imbalance. We also propose a breast-level, two-stage model that separates whether from when. It matches LMV-Net in discrimination (C-index 0.745 vs. 0.746) but orders cancer cases better (ctime 0.58 vs. 0.51, paired 95% CI of the difference 0.01-0.13), on par with Mirai's published weights (0.57).
Create a lesson
Related papers
The long-term divergence exponent and the temporal structure of step-to-step perturbations: a simulation study in two passive walking models
Philippe Terrier
Ab-initio cryo-EM reconstruction from moments of denoised images
Yunpeng Shi, Amit Singer, Yuguan Wang et al.
STORM: A Lightweight Structure-Free Model for Binding Affinity Prediction
Ayush Pandit, Rohan V. Koodli, Wei Deng et al.
Optimising Recruitment and Sample Allocation in Multi-Site Rare Disease Studies
Farid Zare, Catherine Clancy, Tal Dattner et al.
OCCAM: Improved Single-Cell Representation Learning and Perturbation Effect Prediction
Ihab Bendidi, Kian Kenyon-Dean, Alisandra K. Denton et al.
Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model
Yong Liu, Zhanpeng Shi, Peicong Lin et al.