A mechanism of three-dimensional quadratic termination for the gradient method with applications

Abstract

Recent studies show that the two-dimensional quadratic termination property has great potential in improving performance of the gradient method. However, it is not clear whether higher-dimensional quadratic termination leads further benefits. In this paper, we provide an affirmative answer by introducing a mechanism of three-dimensional quadratic termination for the gradient method. A novel stepsize is derived from the mechanism such that a family of delayed gradient methods equipping with the novel stepsize have the three-dimensional quadratic termination property. When applied to the Barzilai--Borwein (BB) method, the novel stepsize does not require the use of any exact line search or the Hessian, and can be computed by stepsizes and gradient norms in previous iterations. Using long BB steps and some short steps associated with the novel stepsize in an adaptive manner, we develop an efficient gradient method for quadratic optimization and further extend it to general unconstrained optimization. Numerical experiments show that the three-dimensional quadratic termination property can significantly improve performance of the BB method, and the proposed method outperforms gradient methods that use stepsizes with the two-dimensional quadratic termination property.