Subgradient Methods for Nonsmooth Sample Average Approximation

Abstract

We study direct subgradient methods for nonsmooth sample average approximation (SAA). Subgradient descent and Nesterov acceleration with constant step sizes attain polynomial and geometric rates matching their smooth counterparts. After $O(\log n)$ full-batch updates, where $n$ is the sample size, the computed solution has the same first-order expansion and limiting distribution as an SAA solution, even when the iterates do not converge. A bound on subgradient increments transfers population regularity to the sample objective, controls the algorithmic trajectory, and identifies the leading random term. The bound covers convex piecewise linear scenario costs, including two-stage stochastic linear programs with fixed recourse. Moreau smoothing displaces the population solution at first order and changes the limiting distribution of the computed solution. An explicit shift of the residual removes the first-order displacement and improves the expansion remainder. For sparse decisions, restricted increment bounds give convergence, support recovery, and expansions for proximal subgradient descent and iterative hard thresholding.