A Central Limit Theorem for Regularized M-Estimators

Abstract

We prove a quantitative central limit theorem for linear functionals of regularized empirical-risk minimizers in the proportional-dimensional regime \(p=O(n)\). The data columns are independent, not necessarily identically distributed, and satisfy a uniform columnwise Poincaré inequality. Under uniform curvature and smoothness assumptions, and for a quadratic regularizer, we show that every nondegenerate statistic \(\sqrt n\,u^\top\hatθ\), centered by its expectation and normalized by its standard deviation, converges to a standard normal random variable in Wasserstein distance, with rate \(O((\log n)^7n^{-1/4})\). The proof is based on moment and stability bounds for the minimizer, a second-order leave-one-out expansion, and a perturbative normal-approximation argument for functions of independent variables. We also prove the variance upper bound \(\Var(u^\top\hatθ)\le C\norm{u}_2^2/n\), identifying the \(\sqrt n\) fluctuation scale.