Model-free Sign Estimation for High-Throughput Screenings

Abstract

In high-throughput screenings, it is common to estimate the effects of many treatments using a small number of independent trials of each. Because little is known about the distributional properties of the measurements from these trials, it is challenging to identify plausible assumptions that can serve as a basis for inferential statistics in this setting. In this article, we develop a method based on minimal assumptions to infer signs of treatment effects (positive or negative). The proposed method controls the number of misestimated signs by using the number of sign disagreements between measurements of the same treatment as a proxy for the number of sign errors. In simulations, the proposed method compares favorably with the Benjamini-Hochberg procedure applied to invalid $p$-values, which is currently considered best practice for many high-throughput screenings. For real data from the L1000 cell-perturbation platform, the proposed method outperforms existing practices, which fail to control error at the nominal level in some cases and are needlessly conservative in others.