Precise Asymptotics for Dense High-Dimensional Huber Regression under Heteroscedasticity

Abstract

We study Huber regression in a heteroscedastic linear model in the proportional regime $n/p \to δ\in (1, \infty)$. Our main result is an exact characterization of the asymptotic empirical distribution of the Huber estimator. This characterization is in terms of three scalar parameters which determine the first-order centre and the asymptotic variance of the Huber estimator. The result shows that with heteroscedastic noise, the first-order centre may contain an additional component along the variance index coefficient. We derive the displacement coefficients that determine this centre, as well as the exact asymptotic variance. The analysis uses the matrix approximate message passing (AMP) framework to jointly track the regression and variance index coefficients, extending the existing framework for homoscedastic M-estimators. We illustrate the implications of this characterization by constructing a test for centre displacement and examining its empirical size and power through simulations.