From Worst-Case to Smoothed Complexity in Continuous Optimization
Daniel Cortild, Coralia Cartis
Abstract
Worst-case complexity bounds often fail to reflect the practical performance of optimization algorithms, while average-case complexity heavily relies on the distribution of the input. In this work, we develop a smoothed-analysis framework for classical continuous optimization algorithms by studying their behaviour on random perturbations of arbitrary functions. Our main contribution is to apply methods suited for non-degenerate functions to perturbations of degenerate functions, thereby recovering the faster best-known rates for the non-degenerate class. We derive smoothed complexity guarantees for several standard methods, including (Accelerated) Gradient Descent, Conjugate Gradient, and Newton-type methods, under various settings including nonconvex ones, and show that these bounds can substantially improve upon their worst-case guarantees, bringing them closer to the practical behaviour of the respective methods.
Create a lesson
Related papers
Optimal Scheduling of Multi-Site AI Data Centers in Day-Ahead Markets
Xinyang Zhou, Guido Cavraro
Minimization of free energy in a Poisson--Boltzmann problem
Shahid Ali, Klemen Bohinc, Angeles Martinez et al.
Lyapunov Theory for Output-to-State Stability of Nonlinear Infinite-Dimensional Systems
Qiaoling Chen, Andrii Mironchenko, Fabian Wirth
Semi-global Convergence and Finite-Sample Error Bounds for Model Predictive Path Integral Control
Christian Fiedler, Giacomo Borghi
When Suspicion Adds Up: A Stochastic Framework for Optimizing Honeypot-Based Insider-Threat Detection
Saharnaz Mehrani, Lujia Zhan, Mohsen Emadikhiav et al.
Hamiltonian Potential Optimization for Partially Observed Stochastic Optimal Control
Christian Bayer, Saifeddine Ben Naamia, Erik von Schwerin et al.