Matrix Concentration Inequalities and Free Probability II. Two-sided Bounds and Applications

Abstract

The first paper in this series introduced a new family of nonasymptotic matrix concentration inequalities that sharply capture the spectral properties of very general random matrices in terms of an associated noncommutative model. These methods achieved matching upper and lower bounds for smooth spectral statistics, but only provided upper bounds for the spectral edges. Here we obtain matching lower bounds for the spectral edges, completing the theory initiated in the first paper. The resulting two-sided bounds enable the study of problems that require an exact determination of the spectral edges to leading order, which is fundamentally beyond the reach of classical matrix concentration inequalities. To illustrate their utility, we develop two general results that explain phase transitions of spectral outliers of a large class of nonhomogeneous random matrices. This enables us to elucidate phase transition phenomena that arise in diverse applications, including decoding node labels on graphs, tensor PCA, contextual stochastic block models, and centered sample covariance matrices.