Confidence Limits and their Robustness
Rajendran Raja
Abstract
Confidence limits are common place in physics analysis. Great care must be taken in their calculation and use, especially in cases of limited statistics when often one-sided limits are quoted. In order to estimate the stability of the confidence levels to addition of more data and/or change of cuts, we argue that the variance of their sampling distributions be calculated in addition to the limit itself. The square root of the variance of their sampling distribution can be thought of as a statistical error on the limit. We thus introduce the concept of statistical errors of confidence limits and argue that not only should limits be calculated but also their errors in order to represent the results of the analysis to the fullest. We show that comparison of two different limits from two different experiments becomes easier when their errors are also quoted. Use of errors of confidence limits will lead to abatement of the debate on which method is best suited to calculate confidence limits.
Create a lesson
Related papers
Reduced latent leakage does not reliably predict lower likelihood bias in collider inference
Tong Pan
The Greedy Bump Bias: Local Profiling Geometry and the Look-Elsewhere Effect
Tommaso Dorigo
Multi-fidelity Monte Carlo estimation of floor response spectra under combined seismic and structural parameter uncertainties
Nils Baillie, Baptiste Kerleguer, Cyril Feau et al.
Parameter inference from a non-stationary unknown process using statistical feature-based slow feature analysis
Kieran S. Owens, Masako Tamaki, Ben D. Fulcher
A Probability Model for Pentagonal Prism Dice Rolls
Paul R. Hurst, J. Naleo Hyde
Geometry-native machine learning reconstruction of DSMC moment fields with support monitoring
Ehsan Roohi