Sampling from high-dimensional, multimodal distributions using automatically tuned, tempered Hamiltonian Monte Carlo

Abstract

Hamiltonian Monte Carlo (HMC) is widely used for sampling from high dimensional target distributions with densities known up to proportionality. While HMC exhibits favorable scaling properties in high dimensions, it struggles with strongly multimodal distributions. Tempering methods are commonly used to address multimodality, but they can be difficult to tune, especially in high dimensional settings. In this study, we propose a method that combines tempering with HMC to enable efficient sampling from high dimensional, strongly multimodal distributions. Our approach simulates the dynamics of a time-varying Hamiltonian in which the temperature increases and then decreases over time. In the first phase, the simulated trajectory gradually explores low-density regions farther from the mode; the second phase guides it back toward a local mode. We develop efficient tuning strategies based on a time-scale transformation under which the Hamiltonian becomes approximately stationary. This leads to a tempered Hamiltonian Monte Carlo (THMC) algorithm with automatic tuning. We demonstrate numerically that our method scales more effectively with dimension than adaptive parallel tempering and tempered sequential Monte Carlo. Finally, we apply our THMC to sample from strongly multimodal posterior distributions arising in Bayesian inference.