Dimension-free Bounds for Sum of Dependent Matrices and Operators with Heavy-Tailed Distribution

Abstract

We prove deviation inequalities for sums of high-dimensional random matrices and operators with dependence and {\rc heavy tails}. Estimation of high-dimensional matrices is a concern for numerous modern applications. However, most results are stated for independent observations. Therefore, it is critical to derive results for dependent and heavy-tailed matrices. In this paper, we derive a dimension-free upper bound on the deviation of the sums. Thus, the bound does not depend explicitly on the dimension of the matrices but rather on their effective rank. Our result generalizes several existing studies on the deviation of sums of matrices. It relies on two techniques: (i) a variational approximation of the dual of moment generating functions, and (ii) robustification through the truncation of the eigenvalues of the matrices. We reveal that our results are applicable to several problems, such as covariance matrix estimation, hidden Markov models, and overparameterized linear regression. At the beginning, we have attached a corrigendum of the original paper. We correct Theorem 4 of the original paper by introducing a log-Sobolev inequality in place of the boundedness condition. We show that the examples discussed in the original paper can be recovered under new conditions. The original paper, uncorrected version -- which includes the aforementioned error -- is appended after this corrigendum for transparency and comparison.