Signal significance incorporating systematic uncertainty for continuous test

Abstract

To properly estimate signal significance while accounting for both statistical and systematic uncertainties, we conducted a study to analyze the impact of typical systematic uncertainties, such as background shape, signal shape, and the number of backgrounds, on significance calculation using the continuous test method. Our investigation reveals unexpected and complex features, leading us to recommend a conservative approach: one should estimate signal significance by conducting trials with as many as possible combinations of various uncertainties associated with the fitting procedure, and then select the "worst" outcome as the final result.