A remark on unified error exponents: Hypothesis testing, data compression and measure concentration

Abstract

Let A be finite set equipped with a probability distribution P, and let M be a "mass" function on A. A characterization is given for the most efficient way in which A^n can be covered using spheres of a fixed radius. A covering is a subset C_n of A^n with the property that most of the elements of A^n are within some fixed distance from at least one element of C_n, and "most of the elements" means a set whose probability is exponentially close to one (with respect to the product distribution P^n). An efficient covering is one with small mass M^n(C_n). With different choices for the geometry on A, this characterization gives various corollaries as special cases, including Marton's error-exponents theorem in lossy data compression, Hoeffding's optimal hypothesis testing exponents, and a new sharp converse to some measure concentration inequalities on discrete spaces.