Convergence of iterates and improved rates for accelerated augmented Lagrangian methods for linearly constrained convex optimization

Abstract

Motivated by an inertial primal-dual dynamical system with vanishing damping, we propose a class of accelerated augmented Lagrangian methods with Nesterov extrapolation parameters for a linearly constrained convex optimization problem with a differentiable objective function. The framework contains two variants: an implicit-gradient scheme for convex continuously differentiable objectives and a partially explicit scheme for convex smooth objectives. Under suitable parameter conditions, we prove convergence of the primal-dual sequence to a primal-dual solution, together with accelerated estimates for the augmented Lagrangian gap, the feasibility violation, and the objective residual. In the noncritical parameter regime, these estimates are improved from $\mathcal{O}(1/k^2)$ to $o(1/k^2)$. Numerical experiments are also presented to illustrate the theoretical results. To the best of our knowledge, neither $o(1/k^2)$ rates for both feasibility violation and objective residual nor convergence of iterates under the critical parameter condition have been previously established for accelerated augmented Lagrangian-type methods in this setting.