Underestimating the uncertainty of aggregated results: the case of W-Boson mass

Abstract

Estimates of uncertainty or variance in experimental means are central to physics. This is especially the case for `world averages' of fundamental physical parameters in particle physics, which aggregate results from a number of experiments to express current knowledge about these parameters and where variance in these world averages reflect uncertainty in that knowledge. The standard aggregation method used to estimate such parameters is a form of hierarchical fixed-effect meta-analysis. This method is inherently unsuitable for estimating these parameters because it ignores sources of random variation between experiments (that is, variation in experimental accuracy, which becomes increasingly important as experimental precision rises). This problem is well-known in the statistical literature, where the typical recommendation is to use random -- rather than fixed-effect techniques. We illustrate this problem by applying random-effect meta-analysis to estimates of the W-Boson mass.