Near Optimality of Discrete-Time Approximations for Controlled McKean-Vlasov and Large Interacting Particle Diffusions

Abstract

We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Via continuity of the expected cost in control policy by endowing the space of relaxed policies with a compact weak topology, we prove near-optimality of piecewise-constant policies which leads to a discrete-time model. We show that the discrete-time value functions (for finite-horizon and discounted infinite-horizon) converge to their continuous-time counterparts as the timestep converges to zero. In particular, we establish that optimal policies for the discrete-time model exists and they are near-optimal for the original continuous-time problem. (ii) We then extend these approximation and near-optimality results to $N$-particle interacting systems under centralized or decentralized mean-field sharing information structure, proving that the discrete-time McKean-Vlasov policy is asymptotically optimal as $N\to \infty$ and the time step goes to zero. Using discrete-time approximation as an intermediate step leads to complementary conditions compared to those in the literature. (iii) We thus develop a unified approximation framework for McKean-Vlasov optimal control problems via discrete-time McKean-Vlasov control problems (and associated numerical methods such as finite model approximations), and we also show near optimality of such approximate policy solutions for the $N$-agent interacting models under centralized and decentralized control.