Further analysis and extension of the higher-order Newton method of Ahmadi, Chaudhry, and Zhang

Abstract

We extend a $d$th-order Newton method for unconstrained optimization by Ahmadi, Chaudhry, and Zhang [Advances in Mathematics, 452:109808] to optimization with SOS-convex polynomial constraints. Consider the problem of minimizing a smooth function $f:\mathbb{R}^{n}\to\mathbb{R}$ subject to SOS-convex polynomial constraints. Given an iterate $x\in\mathbb{R}^n$, Ahmadi et al. define the next iterate $x^{+}$ as the minimizer of the $d$th-order Taylor expansion of $f$ at $x$ with a regularization term of degree $d^{\prime}$, where $d^{\prime}$ is the smallest even number greater than $d$, chosen such that this polynomial is SOS-convex, subject to the constraints. Constructing this polynomial and minimizing it subject to the constraints can both be reduced in time polynomial in $n$ to a semidefinite program (SDP). We prove that, if $f$ is strongly convex and the tensor of the $d$th-order partial derivatives of $f$ is Lipschitz continuous, then our method converges locally to the optimal solution $x^{\ast}$ with order $d$. We further prove that, under certain constraint qualifications, the set of active constraints at $x^{\ast}$ is identified locally in a single iteration. Next, we study the worst-case performance of the third-order Newton method in the unconstrained setting for two classes of univariate $f$ using performance estimation. Finally, we extend a globally convergent modification of the $d$th-order Newton method to the setting of SOS-convex polynomial constraints.