Weighted Randomized Kaczmarz for Sparsely Corrupted Linear Systems

Abstract

In this paper, we introduce a generalized weighted randomized Kaczmarz framework to approximate solutions to overdetermined linear systems whose measurements are perturbed by sparse, additive corruptions. The proposed framework samples equations with probabilities proportional to a user-defined weighting function of their corresponding residual magnitudes. Our work focuses on weighting functions that will reduce the influence of potentially corrupted equations, and provides analyses for a general class of non-increasing weighting functions. We derive one-step expected squared error bounds that separate progress on uncorrupted equations from the contribution of corrupted equations, and provide sufficient conditions for geometric convergence under one specific family of weighting functions -- threshold weighting. We also investigate adaptive exponential weights scaled by the median or median absolute deviation of residual magnitudes, requiring no prior knowledge of the corruption fraction or magnitudes. Lastly, we consider a family of inverse residual weighting functions. Numerical experiments demonstrate the benefit of such a generalized framework in its flexibility in developing approaches for settings in which information about the corruptions is known or unknown.