Unifying restart accelerated gradient and proximal bundle methods

Abstract

This paper presents a novel restarted version of Nesterov's accelerated gradient method and establishes its optimal iteration-complexity for solving convex smooth composite optimization problems. The proposed restart accelerated gradient method is shown to be a specific instance of the accelerated inexact proximal point framework introduced in "An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods" by Monteiro and Svaiter, SIAM Journal on Optimization, 2013. Furthermore, this work examines the proximal bundle method within the inexact proximal point framework, demonstrating that it is an instance of the framework. Notably, this paper provides new insights into the underlying algorithmic principle that unifies two seemingly disparate optimization methods, namely, the restart accelerated gradient and the proximal bundle methods.