A Scalable Frank-Wolfe based Augmented Lagrangian Method for Linearly Constrained Composite Convex Programming

Abstract

In this paper, we consider large-scale linearly constrained composite convex optimization problem, whose objective is a sum of a smooth function and a possibly nonsmooth function. We propose a scalable \textbf{F}rank-\textbf{W}olfe based \textbf{A}ugmented \textbf{L}agrangian (FW-AL) method for solving this problem. At each iteration, the proposed FW-AL method employs the FW method (or its variants) to approximately solve the AL subproblem {(with fixed Lagrange multiplier)} within a preselected tolerance and then updates the Lagrange multiplier. The proposed FW-AL method is well suitable for solving large-scale problems, because its computational cost per step scales (essentially) linearly with the size of the input. We analyze the non-ergodic convergence rate of the proposed FW-AL method.