A Short Review of Estimators for the GLM predictive of Laplace Bayesian Neural Networks

Abstract

This short review examines the primary approaches for estimating the predictive distribution of Laplace-approximated Bayesian neural networks, with particular focus on the Generalized Linear Model (GLM) formulation. We survey the landscape of estimation strategies, from exact GLM computations requiring full Jacobian evaluations to Monte Carlo approximations that trade computational cost for statistical efficiency. The review covers the theoretical foundations of the Laplace approximation, the Kronecker-factored approximate curvature (KFAC) method for scalable posterior inference, and the various predictive estimation techniques developed in the literature. We provide a unified presentation that clarifies the relationships between methods and highlights their respective computational and statistical trade-offs.