A measure concentration effect for matrices of high, higher, and even higher dimension

Abstract

Let $n>m$, and let $A$ be an $(m\times n)$-matrix of full rank. Then obviously the estimate $\|Ax\|\leq\|A\|\|x\|$ holds for the euclidean norm of $x$ and $Ax$ and the spectral norm as the assigned matrix norm. We study the sets of all $x$ for which, for fixed $\delta<1$, conversely $\|Ax\|\geq\delta\,\|A\|\|x\|$ holds. It turns out that these sets fill, in the high-dimensional case, almost the complete space once $\delta$ falls below a bound that depends on the extremal singular values of $A$ and on the ratio of the dimensions. This effect has much to do with the random projection theorem, which plays an important role in the data sciences. As a byproduct, we calculate the probabilities this theorem deals with exactly.