Robust Surrogate-Based Bayesian Inference via Sampling-Based Adaptive Active Learning (SALE)

Abstract

Bayesian inference is difficult when likelihood evaluations are expensive and budgets are limited. We propose sampling-based adaptive active learning (SALE), a Gaussian-process (GP) framework for surrogate-based Bayesian inference. SALE uses the expected posterior (EP) induced by normalised GP sample paths as a common sequential-design measure: it defines a posterior-guided search region and weights uncertainty reduction (UR). A state-dependent rule allocates evaluations between Bayesian optimisation (BO) for localisation and UR for calibration. For BO, an annealed objective interpolates between the EP and Thompson sampling while regularising the query law against surrogate-path perturbations. For UR, we introduce an ideal EP-weighted rule and a computationally feasible proxy. Under a Bayesian GP framework, we characterise the annealed objective's stability--bias trade-off through perturbation and Bayesian regret bounds, derive explicit budget-dependent expected total-variation control for the ideal EP-weighted UR, and establish an expected total-variation rate for the implemented proxy. Across analytic benchmarks and simulated likelihoods, SALE reduces total-variation error across all considered settings while avoiding severe failures seen under several external baselines. Econometric and astrophysical examples demonstrate its practical value.