Enumerable Distributions, Randomness, Dependence
Abstract
Mutual information I in infinite sequences (and in their finite prefixes) is essential in theoretical analysis of many situations. Yet its right definition has been elusive for a long time. I address it by generalizing Kolmogorov Complexity theory from measures to SEMImeasures i.e, infimums of sets of measures. Being concave rather than linear functionals, semimeasures are quite delicate to handle. Yet, they adequately grasp various theoretical and practical scenaria. A simple lower bound i(α:β) = \,x∈ N\,(K(x) - K(x|α) - K(x|β)) for information turns out tight for Martin-Lof random α,β . For all sequences I(α:β) is characterized by the minimum of i(α':β') over random α',β' with U(α')=α, U(β')=β .
Turn this paper into a lesson
ArcXiv compiles a structured reading guide from this paper's metadata: plain-English importance, contributions, prerequisite concepts, which sections to read first, flashcards, and a quiz. Grounded in the abstract, never invented.