On Shore and Johnson properties for a Special Case of Csisz\'ar f-divergences

Abstract

The importance of power-law distributions is attributed to the fact that most of the naturally occurring phenomenon exhibit this distribution. While exponential distributions can be derived by minimizing KL-divergence w.r.t some moment constraints, some power law distributions can be derived by minimizing some generalizations of KL-divergence (more specifically some special cases of Csisz\'ar f-divergences). Divergence minimization is very well studied in information theoretical approaches to statistics. In this work we study properties of minimization of Tsallis divergence, which is a special case of Csisz\'ar f-divergence. In line with the work by Shore and Johnson (IEEE Trans. IT, 1981), we examine the properties exhibited by these minimization methods including the Pythagorean property.