Killed Markov Decision Processes on Finite Time Interval for Countable Models

Abstract

We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is reduced to a similar problem for the derived model. We receive an optimality equation and a method for the construction of simple optimal policies. The sufficiency of simple policies for countable models is proven. We show the correctness of the Markovian property. Additionally, a dynamic programming principle is considered.