A unified theory of the high-dimensional Laplace approximation with application to Bayesian inverse problems

Abstract

The Laplace approximation (LA) to posteriors is a ubiquitous tool to simplify Bayesian computation, particularly in the high-dimensional settings arising in Bayesian inverse problems. Precisely quantifying the LA accuracy is a challenging problem in the high-dimensional regime. We develop a theory of the LA accuracy to high-dimensional posteriors which both subsumes and unifies a number of results in the literature. The primary advantage of our theory is that we introduce a new degree of flexibility, which can be used to obtain problem-specific upper bounds which are much tighter than previous "rigid" bounds. We demonstrate the theory in a prototypical example of a Bayesian inverse problem, in which this flexibility enables us to improve on prior bounds by an order of magnitude. Our optimized bounds in this setting are dimension-free, and therefore valid in arbitrarily high dimensions.