Group invariance of $f$-divergences and the Fisher--Rao distance

Abstract

Many statistical models have natural symmetries described by a group action. We study how such symmetries affect the comparison of two distributions. We work with a transformation model in which a group acts on both the sample space and the parameter space, and the densities transform with a multiplier. Under this assumption, we show that every $f$-divergence is invariant under the group action. As a consequence, an invariant divergence depends only on a maximal invariant of the pair of parameters. When the action on the parameter space is transitive, this maximal invariant is given by a double coset. We apply this result to multidimensional location-scale families, and we show that the same reduction applies to the Fisher--Rao distance.