Recent reproducibility estimates indicate that negative evidence is observed over 30 times before publication

Abstract

The Open Science Collaboration recently reported that 36% of published findings from psychological studies were reproducible by independent researchers. We can use this information together with Bayes theorem to estimate the statistical power needed to produce these findings under various assumptions and calculate the expected distribution of positive and negative evidence for a range of prior probabilities of the tested hypotheses; and by comparing this distribution to other findings indicating that >90% of publications in the psychological literature are statistically significant in support of the authors hypothesis, we can estimate the magnitude of publication bias. The results indicate that negative evidence was observed 30--200 times before one was published.