Matrix Compression using the Nystroöm Method

Abstract

The Nyström method is routinely used for out-of-sample extension of kernel matrices. We describe how this method can be applied to find the singular value decomposition (SVD) of general matrices and the eigenvalue decomposition (EVD) of square matrices. We take as an input a matrix $M\in \mathbb{R}^{m\times n}$, a user defined integer $s\leq min(m,n)$ and $A_M \in \mathbb{R}^{s\times s}$, a matrix sampled from the columns and rows of $M$. These are used to construct an approximate rank-$s$ SVD of $M$ in $O\left(s^2\left(m+n\right)\right)$ operations. If $M$ is square, the rank-$s$ EVD can be similarly constructed in $O\left(s^2 n\right)$ operations. Thus, the matrix $A_M$ is a compressed version of $M$. We discuss the choice of $A_M$ and propose an algorithm that selects a good initial sample for a pivoted version of $M$. The proposed algorithm performs well for general matrices and kernel matrices whose spectra exhibit fast decay.