Near-Optimal Algorithms for Convex Simple Bilevel Optimization under Weak Assumptions

Abstract

This paper considers the simple bilevel optimization (SBO) problem, which minimizes a composite convex function over the optimal solution set of another composite convex minimization problem. We first show that this bilevel problem is equivalent to finding the left-most root of a nonlinear equation. Based on this and a novel dual approach for solving the subproblem in each iteration, we efficiently obtain an $(\epsilon, \epsilon)$-optimal solution through the bisection and Newton methods. The proposed methods achieve near-optimal operation complexity of ${\tilde{\mathcal{O}}(\sqrt{1/\epsilon})}$ under mild assumptions, aligning with the lower complexity bounds of the first-order methods in SBO with both level objectives being smooth convex and unconstrained composite convex optimization when ignoring logarithmic terms.