A Stochastic Gradient Descent Method for Globally Minimizing Nearly Convex Functions

Abstract

This paper proposes a stochastic gradient descent method with an adaptive Gaussian noise term for the global minimization of nearly convex functions, which are nonconvex and possess multiple strict local minimizers. The noise term, independent of the gradient, is determined by the difference between the current function value and a lower bound estimate of the optimal value. In both probability space and state space, we show that the proposed algorithm converges linearly to a neighborhood of the global optimal solution. The size of this neighborhood depends on the variance of the gradient and the deviation between the estimated lower bound and the optimal value. In particular, when full gradient information is available and a sharp lower bound of the objective function is provided, the algorithm achieves linear convergence to the global optimum. Furthermore, we introduce a double-loop scheme that alternately updates the lower bound estimate and the optimization sequence, enabling convergence to a neighborhood of the global optimum that depends solely on the gradient variance. Numerical experiments on several benchmark problems demonstrate the effectiveness of the proposed algorithm.