The case for balanced hypothesis tests and equal-tailed confidence intervals
Abstract
Introduction: there is an ongoing debate about directional inference of two-sided hypothesis tests for which some authors argue that rejecting θ = θ0 does not allow to conclude that θ > θ0 or θ < θ0 but only that θ ≠ θ0, while others argue that this is a minor error without practical consequence. Discussion: new elements are brought to the debate. It is shown that the directional interpretation of some non-directional hypothesis tests about Receiver Operating Characteristic (ROC) and survival curves may lead to inflated type III error rates with a probability of concluding that a difference exists in the opposite side of the actual difference that can reach 50% in the worst case. Some of the issues of directional tests also apply to two-sided confidence intervals (CIs). It is shown that equal-tailed CIs should be preferred to shortest CIs. New assessment criteria of two-sided CIs and hypothesis tests are proposed to provide a reliable directional interpretation: partial left-sided and right-sided α error rates for hypothesis tests, probabilities of overestimation and underestimation αL and αU and interval half-widths for two-sided CIs. Conclusion: two-sided CIs and two-sided tests are interpreted directionally. This implies that directional interpretation be taken in account in the development and evaluation of confidence intervals and tests.