New Error Analysis for Lasso

Abstract

The Lasso is one of the most important approaches for parameter estimation and variable selection in high dimensional linear regression. At the heart of its success is the attractive rate of convergence result even when $p$, the dimension of the problem, is much larger than the sample size $n$. In particular, Bickel et al. (2009) showed that this rate, in terms of the $\ell_1$ norm, is of the order $s\sqrt{(\log p)/n}$ for a sparsity index $s$. In this paper, we obtain a new bound on the convergence rate by taking advantage of the distributional information of the model. Under the normality or sub-Gaussian assumption, the rate can be improved to nearly $s/\sqrt{n}$ for certain design matrices. We further outline a general partitioning technique that helps to derive sharper convergence rate for the Lasso. The result is applicable to many covariance matrices suitable for high-dimensional data analysis.