REACT to NHST: Sensible conclusions to meaningful hypotheses

Abstract

While Null Hypothesis Significance Testing (NHST) remains a widely used statistical tool, it suffers from several shortcomings in its common usage, such as conflating statistical and practical significance, the formulation of inappropriate null hypotheses, and the inability to distinguish between accepting the null hypothesis and failing to reject it. Recent efforts have focused on developing alternatives that address these issues. Despite these efforts, conventional NHST remains dominant in scientific research due to its procedural simplicity and mistakenly presumed ease of interpretation. Our work presents an intuitive alternative to conventional NHST designed to bridge the gap between the expectations of researchers and the actual outcomes of hypothesis tests: $\texttt{REACT}$. $\texttt{REACT}$ not only tackles shortcomings of conventional NHST but also offers additional advantages over existing alternatives. For instance, $\texttt{REACT}$ applies to multiparametric hypotheses and does not require stringent significance-level corrections when conducting multiple tests. We illustrate the practical utility of $\texttt{REACT}$ through real-world data examples.