An Estimand-Focused Approach for AUC Generalization and Cross-Study Benchmarking

Abstract

The area under the ROC curve (AUC) is the standard measure of a biomarker's discriminatory accuracy; however, AUC is rarely treated as a population-specific estimand. When validation cohorts differ from the intended target population in case mix, Naïve AUC estimates can mislead both generalization and cross-study comparison. We develop an estimand-focused framework that anchors biomarker AUC inference to a prespecified target population, aligning with the ICH E9(R1) estimand perspective adapted to discrimination rather than treatment effect. The framework supports two scientific goals: generalizing a study-specific AUC to a clinically relevant target population, and benchmarking AUCs across studies on a common population footing. Methodologically, we extend calibration weighting to the U-statistic formulation of AUC, allowing valid estimation even when the target population is characterized only by summary-level covariate information. This setting is common in biomarker validation, where individual-level target data are often unavailable and existing transportability methods may not be applicable. When patient-level real-world data are accessible, the proposed augmented variants provide double robustness and improved efficiency. We establish asymptotic properties and study their performances through comprehensive simulations. Furthermore, we demonstrate the proposed framework on the POWER trials, evaluating baseline stair-climb power (SCP) as a prognostic marker for 6-month survival in advanced non-small-cell lung cancer (NSCLC). Unlike prior work on transporting model-based predictive accuracy, our framework targets the biomarker-level estimand directly and addresses cross-study comparability - an issue not resolved by current methods.