Revisiting sample size planning for receiver operating characteristic studies: a confidence interval approach with precision and assurance

Abstract

Objectives: Estimation of areas under receiver operating characteristic curves (AUCs) and their differences is a key task in diagnostic studies. We aimed to derive, evaluate, and implement simple sample size formulas for such studies with a focus on estimation rather than hypothesis testing. Materials and Methods: Sample size formulas were developed by explicitly incorporating pre-specified precision and assurance, with precision denoted by the lower limit of confidence interval and assurance denoted by the probability of achieving that lower limit. A new variance function was proposed for valid estimation allowing for unequal variances of observations in the disease and non-disease groups. Performance of the proposed formulas was evaluated through simulation. Results: Closed-form sample size formulas were obtained. Simulation results demonstrated that the proposed formulas produced empirical assurance probability close to the pre-specified assurance probability and empirical coverage probability close to the nominal 95%. Real-world worked examples were presented for illustration. Conclusions: Sample size formulas based on estimation of AUCs and their differences were developed. Simulation results suggested good performance in terms of achieving pre-specified precision and assurance probability. An online calculator for implementing the proposed formulas is openly available at https://dishu.page/calculator/.