Foundations of entropy in complex systems

Abstract

This chapter reviews the foundations of entropy and their extensions to complex systems. We first discuss the relation between Boltzmann's formula, multiplicity, coarse-graining, and Shannon entropy, before introducing generalized entropies such as Rényi, Tsallis, and Burg entropy. We then examine Maxwell--Boltzmann, Bose--Einstein, and Fermi--Dirac statistics, structure-forming systems, sample-space reducing processes, Pólya urns, and nonlinear dynamics. Axiomatic approaches are presented through the Shannon--Khinchin axioms, Tempesta group-composability, Hanel--Thurner asymptotic scaling, Shore--Johnson consistency axioms, and Lieb--Yngvason axioms. Finally, we discuss calibration invariance, Hanel--Thurner--Gell-Mann duality between linear and escort averages, and Kolmogorov--Nagumo averages, showing how the same distribution can arise from different entropies, constraints, or dynamics. These results emphasize that the choice of entropy should be guided by the structure and physical properties of the system.