Geometry of maximum-entropy proofs: stationary points, convexity, Legendre transforms, exponential families

Abstract

This note is a geometric commentary on maximum-entropy proofs. Its purpose is to illustrate the geometric structures involved in such proofs, to explain more in detail why the maximization of the entropy can be turned into the minimization of a potential function, and to show how Lagrange transforms emerge from this. A synopsis of the main functions involved in the proof and of their very different properties is given at the end, together with a brief discussion of exponential families of probabilities, which also appear in the proof.