Universality of estimators for high-dimensional linear models with block dependency

Abstract

We study the universality property of estimators for high-dimensional linear models, which implies that the distribution of estimators is independent of whether the covariates follow a Gaussian distribution. Recent developments in high-dimensional statistics typically require covariates to strictly follow a Gaussian distribution to precisely characterize the properties of estimators. To relax this Gaussianity requirement, the existing literature has examined conditions under which estimators achieve universality. In particular, independence among the elements of the high-dimensional covariates has played a critical role. In this study, we focus on high-dimensional linear models with covariates exhibiting block dependence, where covariate elements can only be dependent within each block, and show that estimators for such models retain universality. Specifically, we prove that the distribution of estimators with Gaussian covariates can be approximated by the distribution of estimators with non-Gaussian covariates having the same moments under block dependence. To establish this result, we develop a generalized Lindeberg principle suitable for handling block dependencies and derive new error bounds for correlated covariate elements. We further demonstrate the universality result across several different estimators.