Some Properties of R\'{e}nyi Entropy over Countably Infinite Alphabets

Abstract

In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties shown parallel those known in the finite alphabet case, while others illustrate a quite different behaviour of R\'enyi entropy in the infinite case. In particular, it is shown that, for any distribution $\mathcal P$ and any $r\in[0,\infty]$, there exists a sequence of distributions $\mathcal{P}_n$ converging to $\mathcal{P}$ with respect to the total variation distance, such that $\lim_{n\to\infty}\lim_{\alpha\to{1+}} H_\alpha(\mathcal{P}_n) = \lim_{\alpha\to{1+}}\lim_{n\to\infty} H_\alpha(\mathcal{P}_n) + r$.