Convergence of dual ascent in non-convex/non-differentiable optimization

Abstract

We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the convex envelope. We also introduce a new augmented version, which is shown to have superior convergence properties, and provide new results even for convex but non-differentiable objective functionals (as well as non-convex). The results are applied to low rank approximation of a given matrix, subject to linear constraints. In particular, letting the linear constraints enforce Hankel structure of the respective matrices, the algorithms can be applied to complex frequency estimation. We provide numerical tests in this setting.