Analysis and Performance of the Barzilai-Borwein Step-Size Rules for Optimization Problems in Hilbert Spaces

Abstract

Due to simplicity, computational cheapness, and efficiency, the Barzilai and Borwein (BB) gradient method has received a significant amount of attention in different fields of optimization. In the first part of this paper, based on spectral analysis, R-linear global convergence for the BB-method is proven for strictly convex quadratic problems posed in infinite-dimensional Hilbert spaces. Then this result is strengthened to R-linear local convergence for a class of twice continuously Fr\'echet-differentiable functions. In the second part, aiming at problems governed by partial differential equations (PDE), the mesh-independent principle is investigated for the BB-method. The applicability of these results is demonstrated for three different types of PDE-constrained optimization problems. Numerical experiments illustrate the theoretical results.