Stochastic optimisation method for estimating large deviations

Abstract

A stochastic optimisation method was recently proposed to efficiently compute large deviation functions, used in statistical physics to characterise the fluctuations of nonequilibrium systems, modelled as Markov processes. The method, called LDSTOP, combines numerical techniques from control theory and machine learning to iteratively construct the "driven process", a controlled version of the Markov process used as a model of nonequilibrium system that realises a given fluctuation or large deviation of that system in an optimal way. Compared to other methods based on spectral approximations, importance sampling, cloning or splitting, LDSTOP is simple, flexible, and scalable, as it works by simulating single trajectories that are gradually guided towards the driven process. Here, we illustrate these advantages by applying the method on the full range of Markov processes considered in applications, namely, discrete-time Markov chains, continuous-time jump processes, and diffusion processes. For each type, we define the objective function to be optimised, explain different options available for representing the driven process (using, e.g., neural networks), and provide implementation details through simple applications. With these contributions, we aim to showcase the method's efficiency, as well as its ease of use when combined with available machine learning packages, such as PyTorch, TensorFlow and JAX, for simulating stochastic processes and solving high-dimensional optimisation problems.