Iteration Complexity of Frank-Wolfe and Its Variants for Bilevel Optimization

Abstract

We study Frank-Wolfe (FW) methods for constrained bilevel optimization when the lower-level problem is solved only approximately, yielding biased and inexact hypergradients. We analyze inexact variants of vanilla FW as well as away-step and pairwise FW, and provide convergence rates in the nonconvex setting under gradient errors. By combining these results with recent bounds on hypergradient errors from iterative and approximate implicit differentiation, we derive overall iteration complexity guarantees for bilevel FW. Experiments on two real-world applications validate the theory and demonstrate practical effectiveness.