Taking Bigger Metropolis Steps by Dragging Fast Variables

Abstract

I show how Markov chain sampling with the Metropolis-Hastings algorithm can be modified so as to take bigger steps when the distribution being sampled from has the characteristic that its density can be quickly recomputed for a new point if this point differs from a previous point only with respect to a subset of 'fast' variables. I show empirically that when using this method, the efficiency of sampling for the remaining 'slow' variables can approach what would be possible using Metropolis updates based on the marginal distribution for the slow variables.