Gibbs-Jaynes Entropy Versus Relative Entropy

Abstract

The maximum entropy formalism developed by Jaynes determines the relevant ensemble in nonequilibrium statistical mechanics by maximising the entropy functional subject to the constraints imposed by the available information. We present an alternative derivation of the relevant ensemble based on the Kullback-Leibler divergence from equilibrium. If the equilibrium ensemble is already known, then calculation of the relevant ensemble is considerably simplified. The constraints must be chosen with care in order to avoid contradictions between the two alternative derivations. The relative entropy functional measures how much a distribution departs from equilibrium. Therefore, it provides a distinct approach to the calculation of statistical ensembles that might be applicable to situations in which the formalism presented by Jaynes performs poorly (such as non-ergodic dynamical systems).