The maximum relative entropy principle

Abstract

We show that the naive application of the maximum entropy principle can yield answers which depend on the level of description, i.e. the result is not invariant under coarse-graining. We demonstrate that the correct approach, even for discrete systems, requires maximization of the relative entropy with a suitable reference probability, which in some instances can be deduced from the symmetry properties of the dynamics. We present simple illustrations of this crucial yet surprising feature in examples of classical and quantum statistical mechanics, as well as in the field of ecology.