On the convergence of BFGS on a class of piecewise linear non-smooth functions

Abstract

The quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method has proven to be very reliable and efficient for the minimization of smooth objective functions since its inception in the 1960s. Recently, it was observed empirically that it also works remarkably well for non-smooth problems when combined with the Armijo-Wolfe line search, but only very limited theoretical convergence theory could be established so far. In this paper, we expand these results by considering convex piecewise linear functions with two pieces that are not bounded below. We prove that the algorithm always terminates in a finite number of iterations, eventually generating an unbounded direction. In other words, in contrast to the gradient method, the BFGS algorithm does not converge to a non-stationary point.