Stochastic smoothing accelerated gradient method for general constrained nonsmooth convex composite optimization

Abstract

We propose a novel stochastic smoothing accelerated gradient (SSAG) method for general constrained nonsmooth convex composite optimization, and analyze the convergence rates. The SSAG method allows various smoothing techniques, and can deal with the nonsmooth term that is not easy to compute its proximal term, or that does not own the linear max structure. To the best of our knowledge, it is the first time to develop a stochastic approximation type method that treats the maximization of finite but numerous nonsmooth convex functions as a stochastic function, which significantly improves the computational efficiency. We prove that the SSAG method can simultaneously achieve the best-known order ${\cal{O}}(\frac{1}{\epsilon})$ of iteration complexity, and the optimal order ${\cal{O}}(\frac{1}{\epsilon^2})$ of $\cal{SFO}$ complexity, using variable sample-size. Numerical results on the application arising from the distributionally robust optimization demonstrate the effectiveness and efficiency of the proposed SSAG method.