Convergence of high-index saddle dynamics for degenerate saddle points on critical manifolds

Abstract

The high-index saddle dynamics (HiSD) method provides a powerful framework for finding saddle points and constructing solution landscapes. While originally derived for nondegenerate critical points, HiSD has demonstrated empirical success in degenerate cases, where the Hessian matrix exhibits zero eigenvalues. However, the mathematical and numerical analysis of HiSD for degenerate saddle points remains unexplored. In this paper, utilizing Morse-Bott functions, we present a rigorous analysis of HiSD for computing degenerate saddle points on a critical manifold. We prove the local convergence of the continuous HiSD and establish the linear convergence rate of the discrete HiSD algorithm. Furthermore, we provide a theoretical explanation for the gradient alignment tendency, revealing that the gradient direction asymptotically aligns with a specific Hessian eigenvector. Our analysis also elucidates the flexibility in selecting the index for HiSD in the context of degenerate saddle points. We validate our analytical results through numerical experiments on neural-network loss landscapes and demonstrate that momentum-accelerated variants of HiSD achieve rapid convergence to degenerate saddle points.