Extremum Problems with Total Variation Distance and their Applications

Abstract

The aim of this paper is to investigate extremum problems with pay-off being the total variational distance metric defined on the space of probability measures, subject to linear functional constraints on the space of probability measures, and vice-versa; that is, with the roles of total variational metric and linear functional interchanged. Utilizing concepts from signed measures, the extremum probability measures of such problems are obtained in closed form, by identifying the partition of the support set and the mass of these extremum measures on the partition. The results are derived for abstract spaces; specifically, complete separable metric spaces known as Polish spaces, while the high level ideas are also discussed for denumerable spaces endowed with the discrete topology. These extremum problems often arise in many areas, such as, approximating a family of probability distributions by a given probability distribution, maximizing or minimizing entropy subject to total variational distance metric constraints, quantifying uncertainty of probability distributions by total variational distance metric, stochastic minimax control, and in many problems of information, decision theory, and minimax theory.