Convergence Analysis of the Upwind Difference Methods for Hamilton-Jacobi-Bellman Equations

Abstract

This paper investigates the convergence properties of the upwind difference scheme for the Hamilton--Jacobi--Bellman (HJB) equation, a central partial differential equation in optimal control theory. First, assuming the existence of a classical solution, we show that the numerical solution converges to the true solution with a first-order rate with respect to the time step. This result complements the square-root rate established in previous studies for viscosity solutions. Second, by exploiting the correspondence between HJB equations and conservation laws, we prove the convergence of the optimal control input. This analysis is crucial for practical applications where the control input is the primary quantity of interest, yet it has rarely been addressed in previous studies. Finally, we confirm the validity of our theoretical results through numerical experiments on typical control problems.