On Some Extensions of Bernstein's Inequality for Self-adjoint Operators

Abstract

We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statistics, signal processing and theoretical computer science. The main feature of our bounds is that, unlike the majority of previous related results, they do not depend on the dimension $d$ of the ambient space. Instead, the dimension factor is replaced by the "effective rank" associated with the underlying distribution that is bounded from above by $d$. In particular, this makes an extension to the infinite-dimensional setting possible. Our inequalities refine earlier results in this direction obtained by D. Hsu, S. M. Kakade and T. Zhang.