Simulation study of estimating between-study variance and overall effect in meta-analyses of mean difference

Abstract

Methods for random-effects meta-analysis require an estimate of the between-study variance, $\tau^2$. The performance of estimators of $\tau^2$ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect. For the effect measure mean difference (MD), we review five point estimators of $\tau^2$ (the popular methods of DerSimonian-Laird, restricted maximum likelihood, and Mandel and Paule (MP); the less-familiar method of Jackson; and a new method (WT) based on the improved approximation to the distribution of the $Q$ statistic by \cite{kulinskaya2004welch}), five interval estimators for $\tau^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, Jackson, and the new WT method), six point estimators of the overall effect (the five related to the point estimators of $\tau^2$ and an estimator whose weights use only study-level sample sizes), and eight interval estimators for the overall effect (five based on the point estimators for $\tau^2$, the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval, a modification of HKSJ, and an interval based on the sample-size-weighted estimator). We obtain empirical evidence from extensive simulations and an example.