Optimal exact tests for composite alternative hypotheses on cross tabulated data

Abstract

We present methodology for constructing exact significance tests for cross tabulated data for "difficult" composite alternative hypotheses that have no natural test statistic. We construct a test for discovering Simpson's Paradox and a general test for discovering positive dependence between two ordinal variables. Our tests are Bayesian extensions of the likelihood ratio test, they are optimal with respect to the prior distribution, and are also closely related to Bayes factors and Bayesian FDR controlling testing procedures.