Global Asymptotic Rates Under Randomization: Gauss-Seidel and Kaczmarz

Abstract

Current performance bounds for randomized iterative methods are often considered tight under per-iteration analyses, yet they are notoriously loose in practice. We derive asymptotic performance bounds that narrow this theory-practice gap, leveraging a new technique for bounding the spectral radii of operators arising in randomized iterations and a connection we establish to Perron-Frobenius theory for noncommutative algebras. The asymptotic analysis also uncovers and quantifies the previously unexplained role of relaxation in improving performance, thereby resolving an open problem posed by Strohmer and Vershynin in 2007.