The sketched landing method for large-scale optimization under orthogonality constraints

Abstract

We propose the \emph{sketched landing method}, a randomized variant of the landing method for optimization under orthogonality constraints. Each landing step consists of the sum of a \emph{normal} component, which reduces infeasibility, and a \emph{tangent} component, which decreases the objective function. Our main contribution is the introduction of low-dimensional random \emph{sketch matrices} to reduce the computational cost of these directions. We consider both dense (Gaussian) and sparse (subsampling) sketch matrices, and show how they reduce the per-iteration cost while preserving convergence guarantees in expectation.