Abstract Dynamic Programming on Partially Ordered Spaces

Abstract

We study abstract dynamic programs on partially ordered spaces, pairing the order-theoretic approach to dynamic programming with topological and metric foundations. We show that readily verifiable forms of topological stability, such as global stability and contractivity of the policy operators, deliver the fundamental optimality properties of dynamic programming together with convergence of value function iteration, Howard policy iteration, and optimistic policy iteration. We also prove that stationary policies dominate nonstationary policy plans under very weak assumptions. Applications include Markov decision processes, structural estimation problems in which maximization and integration are interchanged, optimal stopping without discounting, and Bayesian sequential analysis. For the last two, our results weaken existing assumptions and extend algorithmic guarantees for foundational problems.