Adaptive Algorithms for Nonconvex Bilevel Optimization under P{\L} Conditions

Abstract

Existing methods for nonconvex bilevel optimization (NBO) require prior knowledge of first- and second-order problem-specific parameters (e.g., Lipschitz constants and the Polyak-{\L}ojasiewicz (P{\L}) parameters) to set step sizes, a requirement that poses practical limitations when such parameters are unknown or computationally expensive. We introduce the Adaptive Fully First-order Bilevel Approximation (AF${}^2$BA) algorithm and its accelerated variant, A${}^2$F${}^2$BA, for solving NBO problems under the P{\L} conditions. To our knowledge, these are the first methods to employ fully adaptive step size strategies, eliminating the need for any problem-specific parameters in NBO. We prove that both algorithms achieve $\mathcal{O}(1/\epsilon^2)$ iteration complexity for finding an $\epsilon$-stationary point, matching the iteration complexity of existing well-tuned methods. Furthermore, we show that A${}^2$F${}^2$BA enjoys a near-optimal first-order oracle complexity of $\tilde{\mathcal{O}}(1/\epsilon^2)$, matching the oracle complexity of existing well-tuned methods, and aligning with the complexity of gradient descent for smooth nonconvex single-level optimization when ignoring the logarithmic factors.