The Capacity of Finite-State Channels in the High-Noise Regime

Abstract

This paper considers the derivative of the entropy rate of a hidden Markov process with respect to the observation probabilities. The main result is a compact formula for the derivative that can be evaluated easily using Monte Carlo methods. It is applied to the problem of computing the capacity of a finite-state channel (FSC) and, in the high-noise regime, the formula has a simple closed-form expression that enables series expansion of the capacity of a FSC. This expansion is evaluated for a binary-symmetric channel under a (0,1) run-length limited constraint and an intersymbol-interference channel with Gaussian noise.