WTMAD-4: A Fair Weighting Scheme for GMTKN55

Abstract

The GMTKN55 data set is a collection of standard benchmarks used in molecular quantum chemistry that spans small- and large-molecule thermochemistry, reaction barriers, and non-covalent interactions. Herein, we identify a flaw in the weighted mean absolute deviation (WTMAD) definitions commonly used to quantify performance of various electronic-structure methods for the GMTKN55 set, which under-weight some of its component benchmarks by orders of magnitude. A new WTMAD-4 metric is proposed, based on typical errors observed for a set of ten minimally empirical dispersion-corrected density-functional approximations (DFAs), ensuring fair treatment across all benchmarks. The performance of 115 DFAs is then reassessed using WTMAD-4 and we highlight a literature example where a DFA parametrised by minimising WTMAD-2 underperforms for benchmarks marginalised by that metric.