Adaptive Variance-Reduced Stochastic Sequential (Quadratic Constrained) Quadratic Programming

Abstract

In this paper, we propose an adaptive variance-reduced stochastic sequential (quadratic constrained) quadratic programming (AVR-SS(QC)QP) method for nonlinear constrained optimization. The framework integrates SPIDER-based variance reduction, stochastic quasi-Newton curvature information, merit-function globalization, and adaptive switching between stochastic SQP and SQCQP models. The switching mechanism is governed by a relative constraint model curvature indicator that measures the second-order constraint information neglected by the linearized SQP model. A smoothed hysteresis rule ac?tivates the SQCQP model only when the relative constraint curvature is significant. An exact L1 merit function with Armijo backtracking globalizes the resulting directions, while a coupled stochastic Lyapunov analysis controls the interaction between gradient-estimation error and optimization progress. Under standard smoothness, sampling, regularity, and second-order conditions, we establish almost-sure consistency of the variance-reduced gradient estimator and global convergence of the KKT residual. We further show that, near regular KKT points, the curvature indicator vanishes and the method eventually selects the computationally cheaper SQP model. Numerical experiments on a sparse principal component analysis problem illustrate the practical be?havior of the proposed framework.