Do Modern Difference-in-Differences Estimators Change Published Findings in Business Research?
Davide Proserpio
Abstract
Staggered-adoption difference-in-differences (DiD) designs are a workhorse of empirical research in business-school journals. The two-way fixed effects (TWFE) estimator that typically implements them can place negative weights on heterogeneous treatment effects. We ask whether negative weights change published findings by reanalyzing staggered-adoption specifications in business-school journals with journal-wide public replication archives. Following Hull (2026), we use large language model agents to identify eligible specifications, replicate the published TWFE estimate, and re-estimate each specification with the DiD-averaging estimator of Callaway and Sant'Anna (2021) and the imputation estimator of Borusyak et al. (2024). We re-estimate 355 specifications from 40 papers. Across the 283 DiD-averaging and 306 imputation comparisons that pass Hull's leverage screen, from 28 and 29 papers, TWFE and modern estimates have correlations of 0.61 and 0.70, change sign in 25.1 and 18.0 percent of comparisons, and differ significantly with opposite signs in 10.2 and 7.2 percent. Modern standard errors are 1.61 and 1.30 times those of TWFE. Including the comparisons that fail the screen leaves these figures essentially unchanged. Our results suggest that TWFE and heterogeneity-robust estimators disagree in a non-negligible share of the published staggered designs we re-estimate, and more often than in economics.
Create a lesson
Related papers
Measuring climate credibility? Talk, emissions, and corporate debt composition
Georgios Chortareas, Fangyuan Kou, Alexia Ventouri
The Statistical Cost of Second-Order Orthogonality
Juan Carlos Escanciano, Joël R. Terschuur
Structural Fine-Tuning under Economic Restrictions
Yi Niu, Yuecheng Su, Ethan Yang
Informative Local Average Treatment Effects Using Propensity Scores
Martin Huber, Ying-Ying Lee
Network-Adaptive Instrumental Variables
Yi Niu, Zhengyan Xu
Design-Based Inference for Fuzzy Difference-in-Differences
Pedro Picchetti