Unified Theory of Source Coding: Part I -- Two Terminal Problems

Abstract

Since the publication of Shannon's theory of one terminal source coding, a number of interesting extensions have been derived by researchers such as Slepian-Wolf, Wyner, Ahlswede-Körner, Wyner-Ziv and Berger-Yeung. Specifically, the achievable rate or rate-distortion region has been described by a first order information-theoretic functional of the source statistics in each of the above cases. At the same time several problems have also remained unsolved. Notable two terminal examples include the joint distortion problem, where both sources are reconstructed under a combined distortion criterion, as well as the partial side information problem, where one source is reconstructed under a distortion criterion using information about the other (side information) available at a certain rate (partially). In this paper we solve both of these open problems. Specifically, we give an infinite order description of the achievable rate-distortion region in each case. In our analysis we set the above problems in a general framework and formulate a unified methodology that solves not only the problems at hand but any two terminal problem with noncooperative encoding. The key to such unification is held by a fundamental source coding principle which we derive by extending the typicality arguments of Shannon and Wyner-Ziv. Finally, we demonstrate the expansive scope of our technique by re-deriving known coding theorems. We shall observe that our infinite order descriptions simplify to the expected first order in the known special cases.