On Periodic and Aperiodic Optimal Strategies in Solvency Games

Abstract

Solvency games are a gambling problem on infinite-state Markov decision processes in which the state $n \in \mathbb{N}$ represents an investor's fortune. In every round, the investor chooses an action from a finite action set, and every action yields a distribution over integer-valued gains in an interval $\{-\ell,\ldots,m\}$. The risk-averse investor wants to minimise the probability of eventual ruin (reaching a fortune $\le 0$). It was shown in [Berger et al.] that memoryless deterministic optimal strategies exist, but they are not eventually constant in general. Even in the special case of gains in $\{-2,\ldots,1\}$, the optimal strategy may need to make use of two different actions at arbitrarily high fortunes. We show that optimal strategies in solvency games need not be ultimately periodic in general (thus disproving a 2012 conjecture of Kučera). Already in the case of gains in $\{-3,\ldots,1\}$, it is possible for the optimal strategy to be unique but aperiodic. For gains in $\{-2,\ldots,1\}$, there always exists an ultimately periodic optimal strategy whose tail is constant or alternates between two actions. Finally, we show that the optimal strategy is computable if it is unique. Moreover, (some) optimal strategy can always be computed in the case of gains in $\{-\ell,\ldots,1\}$ for any $\ell \in \mathbb{N}$. Computability in the general case however remains open.