Discovery Sensitivity for a Counting Experiment with Background Uncertainty

Abstract

In Particle Physics, a search for a new signal process is often based on observing a Poisson-distributed number of events, whose mean contains contributions from background and, if it exists, the hypothesised signal. The discovery significance can be expressed as an equivalent number of standard deviations derived from the $p$-value of the background-only hypothesis. To characterise the experimental sensitivity, one may report the median, assuming a nominal signal strength, of the discovery significance. In this paper, approximate expressions for the median significance are derived both when the expected number of background events is known and when the background rate is uncertain but constrained by a Poisson control measurement. The formulae are based on a test statistic using the profile likelihood ratio, and the median significance is approximated using the Asimov data set. Higher-order asymptotic corrections, based on the Barndorff-Nielsen $r^\ast$ statistic, are incorporated for both the observed and expected discovery significance. The validity of the resulting expressions is compared with Monte Carlo results and with other formulae for expected significance often used in particle physics. The higher-order corrections are found to provide meaningful improvements at small event yields. The results are important for obtaining an accurate assessment of the sensitivity of a planned experiment and for the optimal choice of cuts that determine the expected numbers of signal and background events.