More Power by using Fewer Permutations

Abstract

It is conventionally believed that a permutation test should ideally use all permutations. If this is computationally unaffordable, it is believed one should use the largest affordable Monte Carlo sample or (algebraic) subgroup of permutations. We challenge this belief by showing we can sometimes obtain dramatically more power by using a tiny subgroup. As the subgroup is tiny, this simultaneously comes at a much lower computational cost. We exploit this to improve the popular permutation-based Westfall & Young MaxT multiple testing method. We study the relative efficiency in a Gaussian location model, and find the largest gain in high dimensions.