Can the Replication Rate Tell Us About Publication Bias?

Abstract

A leading explanation for widespread replication failures is publication bias. I show in a simple model of selective publication that, contrary to common perceptions, the replication rate is unaffected by the suppression of insignificant results in the publication process. I show further that the expected replication rate falls below intended power owing to issues with common power calculations. I empirically calibrate a model of selective publication and find that power issues alone can explain the entirety of the gap between the replication rate and intended power in experimental economics. In psychology, these issues explain two-thirds of the gap.