R\'enyi Cross-Entropy Measures for Common Distributions and Processes with Memory
Abstract
Two R\'enyi-type generalizations of the Shannon cross-entropy, the R\'enyi cross-entropy and the Natural R\'enyi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we build upon our results in [1] by deriving the R\'enyi and Natural R\'enyi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family and tabulating the results for ease of reference. We also summarise the R\'enyi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.
Turn this paper into a lesson
ArcXiv compiles a structured reading guide from this paper's metadata: plain-English importance, contributions, prerequisite concepts, which sections to read first, flashcards, and a quiz. Grounded in the abstract, never invented.