On the exact Berk-Jones statistics and their p-value calculation

Abstract

Continuous goodness-of-fit testing is a classical problem in statistics. Despite having low power for detecting deviations at the tail of a distribution, the most popular test is based on the Kolmogorov-Smirnov statistic. While similar variance-weighted statistics, such as Anderson-Darling and the Higher Criticism statistic give more weight to tail deviations, as shown in various works, they still mishandle the extreme tails. As a viable alternative, in this paper we study some of the statistical properties of the exact $M_n$ statistics of Berk and Jones. We derive the asymptotic null distributions of $M_n, M_n^+, M_n^-$, and further prove their consistency as well as asymptotic optimality for a wide range of rare-weak mixture models. Additionally, we present a new computationally efficient method to calculate $p$-values for any supremum-based one-sided statistic, including the one-sided $M_n^+,M_n^-$ and $R_n^+,R_n^-$ statistics of Berk and Jones and the Higher Criticism statistic. We illustrate our theoretical analysis with several finite-sample simulations.