On the Computation of the Shannon Capacity of a Discrete Channel with Noise

Abstract

Muroga [M52] showed how to express the Shannon channel capacity of a discrete channel with noise [S49] as an explicit function of the transition probabilities. His method accommodates channels with any finite number of input symbols, any finite number of output symbols and any transition probability matrix. Silverman [S55] carried out Muroga's method in the special case of a binary channel (and went on to analyse "cascades" of several such binary channels). This article is a note on the resulting formula for the capacity C(a, c) of a single binary channel. We aim to clarify some of the arguments and correct a small error. In service of this aim, we first formulate several of Shannon's definitions and proofs in terms of discrete measure-theoretic probability theory. We provide an alternate proof to Silverman's, of the feasibility of the optimal input distribution for a binary channel. For convenience, we also express C(a, c) in a single expression explicitly dependent on a and c only, which Silverman stopped short of doing.