The Ces\`aro Value Iteration

Abstract

In this paper, we consider undiscouted infinite-horizon optimal control for deterministic systems with an uncountable state and input space. We specifically address the case when the classic value iteration does not converge. For such systems, we use the Ces`aro mean to define the infinite-horizon optimal control problem and the corresponding infinite-horizon value function. Moreover, for this value function, we introduce the Ces\`aro value iteration and prove its convergence for the special case of systems with periodic optimal operating behavior. For this instance, we also show that the Ces\`aro value function recovers the undiscounted infinite-horizon optimal cost, if the latter is well-defined.