Loss Functions for Measuring the Accuracy of Nonnegative Cross-Sectional Predictions

Abstract

Measuring the accuracy of cross-sectional predictions is a subjective problem. Generally, this problem is avoided. In contrast, this paper confronts subjectivity up front by eliciting an impartial decision-maker's preferences. These preferences are embedded into an axiomatically-derived loss function, one of the simplest version of which is described. The parameters of the loss function can be estimated by linear regression. Specification tests for this function are described. This framework is extended to weighted averages of estimates to find the optimal weightings. A special case occurs when the predictions represent resource allocations: the apportionment literature is used to construct the Webster-Saint Lag\"ue Rule, a particular parametrization of the loss function. These loss functions are compared to those existing in the literature. Finally, a family of bias measures are created using signed versions of these loss functions.