Shannon Information Theory Without Shedding Tears Over Delta \& Epsilon Proofs or Typical Sequences

Abstract

This paper begins with a discussion of integration over probability types (p-types). After doing that, the paper re-visits 3 mainstay problems of classical (non-quantum) Shannon Information Theory (SIT): source coding without distortion, channel coding, and source coding with distortion. The paper proves well-known, conventional results for each of these 3 problems. However, the proofs given for these results are not conventional. They are based on complex integration techniques (approximations obtained by applying the method of steepest descent to p-type integrals) instead of the usual delta & epsilon and typical sequences arguments. Another unconventional feature of this paper is that we make ample use of classical Bayesian networks (CB nets). This paper showcases some of the benefits of using CB nets to do classical SIT.