Refiltering hypothesis tests to control sign error

Abstract

A common, though not recommended statistical practice is to report confidence intervals if and only if they exclude a null value of 0. The resulting filtered confidence intervals generally do not have their nominal confidence level. More worryingly, in low power settings their center points will be much farther from zero than the true parameter is and they will frequently lie on the wrong side of zero. Many confidence intervals are constructed using an asymptotically Gaussian parameter estimate accompanied by a weakly consistent estimate of its variance. In these cases, we can subject the given confidence interval(s) to a second filtering step such that the probability of a sign error is controled. This refiltering step retains only those confidence intervals that are sufficiently well separated from the origin. It requires no assumptions on the dependencies among the test statistics.