On Convergence of Regularized Barzilai-Borwein Method

Abstract

The regularized Barzilai-Borwein (RBB) method represents a promising gradient-based optimization algorithm. In this paper, by splitting the gradient into two parts and analyzing the dynamical system of difference equations governing the ratio of their magnitudes, we establish that the RBB method achieves R-linear convergence for strongly convex quadratic functions of arbitrary dimensions. Specifically, for the two-dimensional case, we provide a concise proof demonstrating that the method exhibits at least R-linear convergence. We propose a simple yet effective adaptive regularization parameter scheme to further improve its performance. A typical numerical example verifies the effectiveness of this scheme.