Efficient estimation of autocorrelation spectra

Abstract

The performance of Markov chain Monte Carlo calculations is determined by both ensemble variance of the Monte Carlo estimator and autocorrelation of the Markov process. In order to study autocorrelation, binning analysis is commonly used, where the autocorrelation is estimated from results grouped into bins of logarithmically increasing sizes. In this paper, we show that binning analysis comes with a bias that can be eliminated by combining bin sizes. We then show binning analysis can be performed on-the-fly with linear overhead in time and logarithmic overhead in memory with respect to the sample size. We then show that binning analysis contains information not only about the integrated effect of autocorrelation, but can be used to estimate the spectrum of autocorrelation lengths, yielding the height of phase space barriers in the system. Finally, we revisit the Ising model and apply the proposed method to recover its autocorrelation spectra.