First-Order Projected Algorithms With the Same Linear Convergence Rate Bounds as Their Unconstrained Counterparts

Abstract

In this paper, we propose a systematic approach for extending first-order optimization algorithms, originally designed for unconstrained strongly convex problems, to handle closed convex set constraints. We show that the resulting projected algorithms retain the same linear convergence rate bounds, provided that the underlying unconstrained optimization algorithms admit a quadratic Lyapunov function obtained from integral quadratic constraint (IQC) analysis. The projected algorithms are constructed by applying a projection in the norm induced by the Lyapunov matrix, ensuring both constraint satisfaction and optimality at the fixed point. Furthermore, under a linear transformation associated with this matrix, the projection becomes non-expansive in the Euclidean norm, thereby preserving the convergence rate bounds under the composition of the linearly convergent algorithmic operator and the projection. Our results indicate that, when analyzing worst-case convergence rates or when synthesizing first-order optimization algorithms with potentially higher-order dynamics, it suffices to focus solely on the unconstrained dynamics, since the same parameters or stepsizes can be employed without retuning.