A Moreau-Yosida approximation scheme for a class of high-dimensional posterior distributions

Abstract

Exact-sparsity inducing prior distributions in Bayesian analysis typically lead to posterior distributions that are very challenging to handle by standard Markov Chain Monte Carlo (MCMC) methods, particular in high-dimensional models with large number of parameters. We propose a methodology to derive smooth approximations of such posterior distributions that are, in some cases, easier to handle by standard MCMC methods. The approximation is obtained from the forward-backward approximation of the Moreau-Yosida regularization of the negative log-density. We show that the derived approximation is within $O(\sqrt{\gamma})$ of the true posterior distribution in the $\beta$-metric, where $\gamma>0$ is a user-controlled parameter that defines the approximation. We illustrate the method with a high-dimensional linear regression model.