Empirical Bayes learning from selectively reported confidence intervals

Abstract

We develop a statistical framework for empirical Bayes learning from selectively reported confidence intervals, and apply it to provide context for interpreting results published in MEDLINE abstracts. We use a collection of 326,060 z-scores from MEDLINE abstracts (2000-2018) as the input for an empirical Bayes analysis, with publication bias as a key methodological challenge. We address publication bias through a selective tilting approach that extends empirical Bayes confidence intervals to truncated sampling. Our framework provides coverage guarantees for functionals including posterior estimands describing idealized replications and the symmetrized posterior mean, which we justify decision-theoretically as optimal among sign-equivariant (odd) estimators.