Inexactly Smooth Performance Estimation and New Optimized Gradient Methods

Abstract

We consider a general class of ``inexactly smooth'' convex functions, providing a universal model capturing as special cases $L$-smooth, $M$-Lipschitz, and Hölder smooth functions, and any combination thereof. Such functions possess a calculus closely following that of smooth functions. Our main results provide inexactly smooth functions with interpolation theorems that are necessary and sufficient up to modest universal constants. These enable analysis of first-order methods for any inexactly smooth convex problem class via solving convex Performance Estimation Problems (PEPs). Further, these enable the extension of Drori and Taylor's constructive approach to algorithm design. From this, we derive an exactly minimax optimal method for $(β,0)$-Hölder smooth problems, methods with the best-known convergence guarantees up to constants for any $(β,p)$-Hölder smooth convex minimization, and a new universal fast backtracking method for any inexactly smooth convex problem.