Binary Hypothesis Testing: A Robust Framework Against the Look Elsewhere Effect

Abstract

In particle physics, discovery claims conventionally require an observed significance exceeding $5σ$. However, the interpretation of a $5σ$ result depends critically on the testing procedure, namely whether the hypothesis is tested at a single pre-specified point in parameter space or by scanning over a range of possible signal locations. This distinction gives rise to the look-elsewhere effect, a concept that is widely used but often interpreted as a simple penalty for scanning. In this work, we reinterpret the look-elsewhere effect as a correction for procedural inconsistency arising when the null distribution is generated under one procedure while the test statistic is evaluated under another. Within the framework of hypothesis testing, we examine its implications and clarify the distinct roles of the look-elsewhere effect in binary and peak-search scenarios. In particular, we show that binary test is robust against the look-elsewhere effect, whereas peak searches require an explicit correction for the search over signal locations. Using a moderate trial factor of approximately 26, calibrated from the ATLAS Higgs search, we show that a $3σ$ global significance of peak search can correspond to approximately $4σ$ significance of the binary test for the same value of the observed test statistic. This reformulation provides a clearer statistical interpretation of the look-elsewhere effect and offers a more coherent framework for understanding significance claims in particle physics.