A Parameterized Barzilai-Borwein Method via Interpolated Least Squares

Abstract

The Barzilai-Borwein (BB) method is an effective gradient descent algorithm for solving unconstrained optimization problems. Based on the observation of two classical BB step sizes, by constructing an interpolated least squares model, we propose a novel class of BB step sizes, each of which still retains the quasi-Newton property, with the original two BB step sizes being their two extreme cases. We present the mathematical principle underlying the adaptive alternating BB (ABB) method. Based on this principle, we develop a class of effective adaptive interpolation parameters. For strictly convex quadratic optimization problems, we establish the R-linear convergence of this new gradient descent method by investigating the evolution pattern of the ratio of the absolute values of the gradient components. Numerical experiments are conducted to illustrate our findings.