On misconceptions about the Brier score in binary prediction models

Abstract

The Brier score is a widely used metric evaluating overall performance of probabilistic predictions for binary outcomes in clinical research. However, its interpretation can be complex, as it does not align with commonly taught concepts in medical statistics. Consequently, the Brier score is often misinterpreted, sometimes to a significant extent, a fact that has not been adequately addressed in the literature. We aim to explore prevalent misconceptions surrounding the Brier score and elucidate the understanding and interpretation of Brier scores for practitioners.