How Much Does Random Rotation Help GPTQ? Sharp Worst-Case Rate Gaps in the High-Rate Regime

Abstract

GPTQ is Babai's nearest-plane algorithm on the lattice of the Cholesky factor of the layer's input second moment $Σ$. In the high-rate, entropy-coded regime it needs $\tfrac12\log_2(\AM/\GM)$ of its Cholesky pivots more bits than WaterSIC, which is within $0.255$ bit of the rate-distortion limit for every $Σ$; random rotations close most of this gap in practice, and Ordentlich and Polyanskiy asked for the worst-case gap between rotated GPTQ and WaterSIC. We show that after a Haar rotation the $k$-th pivot converges almost surely, uniformly in $k\geδn$ and over spectra $ν$ of bounded condition number, to $1/S_ν(-k/n)$, a classical formula from the theory of Lyapunov exponents, and use it to answer the question. Over spectra with condition number $κ$ the worst rotated gap is $\tfrac12\log_2κ-\log_2\lnκ+\tfrac12-\tfrac1{\ln2}+o(1)$ bits, attained only by two-point spectra, just $\tfrac12\log_2\lnκ+O(1)$ bits below the worst case without rotation, so rotation cannot replace waterfilling in general. It nearly can on real layers: rotation cuts the cost of an $η$-fraction of outlier eigenvalues of size $μ$ from about $ημ$ to $η^2μ/2$ to leading order, but cannot remove spectral spread on a logarithmic scale; for non-Gaussian weights it also adds a weight-side rate cost, which we characterize exactly under a standard high-rate assumption. On $96$ Hessians of Qwen3 and Llama-3.1 the rotated gap has median $0.04$ bit and maximum $0.36$ bit, against $2$--$5$ bits in the worst case for the same condition numbers, and the limit matches it to within $0.009$ bit.