Generalized Alpha Investing: Definitions, Optimality Results, and Application to Public Databases

Abstract

The increasing prevalence and utility of large, public databases necessitates the development of appropriate methods for controlling false discovery. Motivated by this challenge, we discuss the generic problem of testing a possibly infinite stream of null hypotheses. In this context, Foster and Stine (2008) suggested a novel method named Alpha Investing for controlling a false discovery measure known as mFDR. We develop a more general procedure for controlling mFDR, of which Alpha Investing is a special case. We show that in common, practical situations, the general procedure can be optimized to produce an expected reward optimal (ERO) version, which is more powerful than Alpha Investing. We then present the concept of quality preserving databases (QPD), originally introduced in Aharoni et al. (2011), which formalizes efficient public database management to simultaneously save costs and control false discovery. We show how one variant of generalized alpha investing can be used to control mFDR in a QPD and lead to significant reduction in costs compared to naive approaches for controlling the Family-Wise Error Rate implemented in Aharoni et al. (2011).