When models fail: an introduction to posterior predictive checks and model misspecification in gravitational-wave astronomy

Abstract

Bayesian inference is a powerful tool in gravitational-wave astronomy. It enables us to deduce the properties of merging compact-object binaries and to determine how these mergers are distributed as a population according to mass, spin, and redshift. As key results are increasingly derived using Bayesian inference, there is increasing scrutiny on Bayesian methods. In this review, we discuss the phenomenon of \textit{model misspecification}, in which results obtained with Bayesian inference are misleading because of deficiencies in the assumed model(s). Such deficiencies can impede our inferences of the true parameters describing physical systems. They can also reduce our ability to distinguish the "best fitting" model: it can be misleading to say that Model~A is preferred over Model~B if both models are manifestly poor descriptions of reality. Broadly speaking, there are two ways in which models fail: models that fail to adequately describe the data (either the signal or the noise) have misspecified likelihoods. Population models -- designed, for example, to describe the distribution of black hole masses -- may fail to adequately describe the true population due to a misspecified prior. We recommend tests and checks that are useful for spotting misspecified models using examples inspired by gravitational-wave astronomy. We include companion python notebooks to illustrate essential concepts.