Stability of the Shannon--McMillan--Breiman Theorem under Sublinear Parsings

Abstract

We establish a stability result for the Shannon-McMillan-Breiman theorem on the one-sided finite shift space. For any shift-invariant probability measure P and any data-dependent parsing whose number of blocks is sublinear in N almost surely, we show that the normalized sum of the negative log-likelihoods of the parsing blocks converges almost surely and in L^1(P) to the entropy-rate function h_P. Equivalently, we obtain an approximate factorization of cylinder probabilities under arbitrary sublinear parsings. We further show that the stability result persists under subextensive perturbations of the parsing blocks, and that sublinearity of the block count is the sharp threshold for validity at this level of generality, via a direct counterexample.