Bounding the Error of Value Functions in Sobolev Norm Yields Bounds on Suboptimality of Controller Performance

Abstract

Optimal feedback controllers for nonlinear systems can be derived by solving the Hamilton-Jacobi-Bellman (HJB) equation. However, because the HJB is a nonlinear partial differential equation, numerical methods typically provide only approximate solutions. While numerical error bounds on approximate HJB solutions are often available, these bounds do not necessarily translate into guarantees on the suboptimality of the resulting controllers. In this paper, we establish that the suboptimality of the resulting controller is bounded by the $L^\infty$ norm of the HJB residual, which is, in turn, bounded by numerical error in the value function as measured in the Sobolev $W^{1,\infty}$ norm. This implies that convergence of value functions in $W^{1,\infty}$ result in controllers that yield a cost that is arbitrarily close to the true minimum. In contrast, we demonstrate that such guarantees do not hold when the value function error is measured in weaker norms, such as the Sobolev $W^{1,p}$ norm for finite $p$. These results apply to systems governed by Lipschitz continuous dynamics over a finite time horizon with compact input space.