Analysis of a zeroth-order multiscale sampler: strong convergence, convergence of ergodic averages and weak convergence

Abstract

In many practical applications of Bayesian inference, it is difficult or undesirable to compute the gradient of the negative log-posterior, motivating the study of derivative-free samplers, also called zeroth-order samplers. The aim of this paper is to generalize and rigorously analyze a multiscale derivative-free sampler recently proposed in the literature by Pavliotis, Stuart and Vaes (2022). In the discrete-time setting, the multiscale sampler we study bears a strong resemblance to other recently proposed methods, such as the so-called zeroth-order perturbed stochastic gradient descent algorithm (Jin et al., 2018) and the zeroth-order Langevin Monte Carlo algorithm by Roy et al. (2022). We present error estimates with respect to all the parameters of the problem, both in the continuous-time and discrete-time settings. The error estimates are in three different metrics: strong convergence, convergence of ergodic averages, and uniform-in-time weak convergence.