A Gradient Sampling method based on Ideal direction for solving nonsmooth nonconvex optimization problems: convergence analysis and numerical experiments

Abstract

In this paper, a modification to the Gradient Sampling (GS) method for minimizing nonsmooth nonconvex functions is presented. One drawback in GS method is the need of solving a Quadratic optimization Problem (QP) at each iteration, which is time-consuming especially for large scale objectives. To resolve this difficulty, we propose a new descent direction, namely Ideal direction, for which there is no need to consider any quadratic or linear optimization subproblem. It is shown that, this direction satisfies Armijo step size condition and can be used to make a substantial reduction in the objective function. Furthermore, we prove that using Ideal directions preserves the global convergence of the GS method. Moreover, under some moderate assumptions, we present an upper bound for the number of serious iterations. Using this upper bound, we develop a different strategy to study the convergence of the method. We also demonstrate the efficiency of the proposed method using small, medium and large scale problems in our numerical experiments.