Headline Estimation with Multiple Research Designs

Abstract

To study a scalar parameter, a researcher may consider multiple research designs. Based on the evidence across designs, the researcher may wish to formulate a headline estimate of the parameter. I examine how to choose this headline when it is unclear which design is most appropriate for studying the parameter. I model this setting by assuming that (i) exactly one of the designs is valid for the parameter and (ii) the researcher has ambiguity about which design is valid, represented by a class of priors over the candidate designs. To account for ambiguity, I propose reporting the headline estimate that minimizes the worst-case posterior risk over the class of priors. In three applications, I show cases where accounting for ambiguity materially affects the quantitative conclusion and cases where an existing headline is already close to optimal.