Kelly Bets and Single-Letter Codes: Optimal Information Processing in Natural Systems

Abstract

In an information-processing investment game, such as the growth of a population of organisms in a changing environment, Kelly betting maximizes the expected log rate of growth. In this paper, we show that Kelly bets are closely related to optimal single-letter codes (i.e., they can achieve the rate-distortion bound with equality). Thus, natural information processing systems with limited computational resources can achieve information-theoretically optimal performance. We show that the rate-distortion tradeoff for an investment game has a simple linear bound, and that the bound is achievable at the point where the corresponding single-letter code is optimal. This interpretation has two interesting consequences. First, we show that increasing the organism's portfolio of potential strategies can lead to optimal performance over a continuous range of channels, even if the strategy portfolio is fixed. Second, we show that increasing an organism's number of phenotypes (i.e., its number of possible behaviours in response to the environment) can lead to higher growth rate, and we give conditions under which this occurs. Examples illustrating the results in simplified biological scenarios are presented.