Loss functions arising from the index of agreement

Abstract

We examine the theoretical properties of the index of agreement loss function $L_W$, the negatively oriented counterpart of Willmott's index of agreement, a common metric in environmental sciences and engineering. We prove that $L_W$ is bounded within [0, 1], translation and scale invariant, and estimates the parameter $\Bbb{E}_{F}[\underline{y}] \pm \Bbb{V}_{F}^{1/2}[\underline{y}]$ when fitting a distribution. We propose $L_{\operatorname{NR}_2}$ as a theoretical improvement, which replaces the denominator of $L_W$ with the sum of Euclidean distances, better aligning with the underlying geometric intuition. This new loss function retains the appealing properties of $L_W$ but also admits closed-form solutions for linear model parameter estimation. We show that as the correlation between predictors and the dependent variable approaches 1, parameter estimates from squared error, $L_{\operatorname{NR}_2}$ and $L_W$ converge. This behavior is mirrored in hydrologic model calibration (a core task in water resources engineering), where performance becomes nearly identical across these loss functions. Finally, we suggest potential improvements for existing $L_p$-norm variants of the index of agreement.