Faster Low-rank Approximation using Adaptive Gap-based Preconditioning

Abstract

We propose a method for rank $k$ approximation to a given input matrix $X \in \mathbb{R}^{d \times n}$ which runs in time \[ \tilde{O} \left(d ~\cdot~ \min\left\{n + \tilde{sr}(X) \,G^{-2}_{k,p+1}\ ,\ n^{3/4}\, \tilde{sr}(X)^{1/4} \,G^{-1/2}_{k,p+1} \right\} ~\cdot~ \text{poly}(p)\right) ~, \] where $p>k$, $\tilde{sr}(X)$ is related to stable rank of $X$, and $G_{k,p+1} = \frac{\sigma_k-\sigma_p}{\sigma_k}$ is the multiplicative gap between the $k$-th and the $(p+1)$-th singular values of $X$. In particular, this yields a linear time algorithm if the gap is at least $1/\sqrt{n}$ and $k,p,\tilde{sr}(X)$ are constants.