Heavy Ball and Nesterov Accelerations with Hessian-driven Damping for Nonconvex Optimization

Abstract

In this work, we investigate a second-order dynamical system with Hessian-driven damping tailored for a class of nonconvex functions called strongly quasiconvex. Buil\-ding upon this continuous-time model, we derive two discrete-time gra\-dient-based algorithms through time discretizations. The first is a Heavy Ball method with Hessian correction, incorporating cur\-va\-tu\-re-dependent terms that arise from discretizing the Hessian damping component. The second is a Nesterov-type accelerated method with adaptive momentum, fea\-tu\-ring correction terms that account for local curvature. Both algorithms aim to enhance stability and convergence performance, particularly by mi\-ti\-ga\-ting oscillations commonly observed in cla\-ssi\-cal momentum me\-thods. Furthermore, in both cases we establish li\-near convergence to the optimal solution for the iterates and functions values. Our approach highlights the rich interplay between continuous-time dynamics and discrete optimization algorithms in the se\-tting of strongly quasiconvex objectives. Numerical experiments are presented to support obtained results.