Entropy Estimates from Stochastic Interpolants

Abstract

We present a general method for estimating entropy differences between arbitrary probability distributions using stochastic interpolants. The marginal distributions which bridge the base and target obey a continuity equation, which allows a straightforward calculation of the entropy difference in terms of an inner product of the probability flow velocity and score fields. This formulation has several advantages: (i) no computationally expensive divergence calculations of either field are required, (ii) the score field need not even be learned directly if model transferability is not required, and (iii) on-the-fly estimates are produced nearly for free during training. When tractable base distributions are chosen (e.g. Gaussian chain, ideal gas, etc.), statistical thermodynamic entropies are then immediately recovered. Notably, the analysis relies only on the definition of the Gibbs-Shannon entropy, rather than any particular statistical mechanical ensemble, so that generalized non-equilibrium entropies may be computed. The method is demonstrated on several systems of increasing complexity: (i) a 40-dimensional Gaussian mixture model, (ii) the classical XY model of N spins arranged in one dimension, (iii) the 13-atom Lennard-Jones cluster, and (iv) an active Brownian polymer.