A Note on the Equivalence of Gibbs Free Energy and Information Theoretic Capacity

Abstract

The minimization of Gibbs free energy is based on the changes in work and free energy that occur in a physical or chemical system. The maximization of mutual information, the capacity, of a noisy channel is determined based on the marginal probabilities and conditional entropies associated with a communications system. As different as the procedures might first appear, through the exploration of a simple, "dual use" Ising model, it is seen that the two concepts are in fact the same. In particular, the case of a binary symmetric channel is calculated in detail.