IT formulae for gamma target: mutual information and relative entropy

Abstract

In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the derivative of the input-output mutual information of this non-linear channel with respect to the channel quality parameter. This relation is reminiscent of the well-known link between the derivative of the input-output mutual information of additive Gaussian noise channel with respect to the signal-to-noise ratio and the minimum mean-square error. The proof relies on a rescaled version of De Bruijn identity for gamma target distribution together with a stochastic representation for the gamma specific Fisher information. Finally, we are able to derive precise bounds and asymptotics for the input-output mutual information of the non-linear channel with gamma inputs.