Adaptation and Development of Super Schemes for Unconstrained Optimization Problems

Abstract

In this paper, we propose a class of super-schemes for efficiently solving nonlinear unconstrained optimization problems. The proposed approach introduces two novel choices of step-size parameters, leading to efficient descent directions without requiring second-order information. We develop one-step, two-step, and three-step iterative schemes (denoted by SS1, SS2, and SS3) and establish that these methods achieve higher-order convergence of orders two, four, and six, respectively. Despite their high convergence rates, the computational complexity of the proposed methods remains comparable to existing gradient-based methods, with a cost of $\mathcal{O}(n^2)$ per iteration. The proposed methods are simple to implement and do not require complicated line-search procedures. Their effectiveness is demonstrated through extensive numerical experiments on a wide range of problems, including large-scale and ill-conditioned cases. The results show that the proposed methods significantly outperform classical methods, such as the Barzilai-Borwein method and other gradient-based approaches, in terms of iteration count and computational efficiency. Finally, the numerical results are consistent with the theoretical analysis, confirming the stability of the proposed schemes for test optimization problems.