Relaxation Kernel, Spectral Dissipation, and Global Convergence of Blahut--Arimoto Dynamics

Abstract

We develop a spectral theory for continuous- and discrete-time Blahut--Arimoto (BA) dynamics, centered on the relaxation kernel $ \G = \E_p[K^*_X \otimes K^*_X] $. Five main results are established. (i) Along the continuous-time BA flow, the free energy satisfies the exact $ χ^2 $-dissipation identity $ \dot F_β= -\D(q) $, where $ \D(q)=χ^2(\T q \| q) $ is the Pearson $ χ^2 $-divergence. (ii) The operator $ \G $ admits a threefold identity: it is simultaneously the Gram matrix of the equilibrium Gibbs kernels, the linearised generator of the BA vector field, and the Fisher--Rao Hessian of the free energy at the fixed point. (iii) For the discrete iteration, the one-step Lyapunov dissipation decomposes spectrally as $ Δ\mathcal{L}^{(2)} = \sum_i c_i^2\, d(λ_i) $, where $ d(λ) = -λ+ \tfrac{3}{2}λ^2 - \tfrac{1}{2}λ^3 $. This reveals a double bottleneck at $ λ\approx 0 $ and $ λ\approx 1 $, with optimal dissipation near $ λ\approx 0.423 $. (iv) Global convergence follows a two-stage mechanism: $ χ^2 $-dissipation drives finite-time entry into a local neighbourhood, after which the spectral gap $ \lam = λ_{\min}(\G|_T) $ governs exponential contraction. (v) The KL convergence factor is explicit: $ \KL(q^*\|q_{n+1}) \le (1-\lam)^2\,\KL(q^*\|q_n) + O(\|v_n\|_*^3) $, with per-iteration improvement $ γ= \lam(2-\lam) $. For Gaussian sources, $ \lam = 1/(2βσ^2) $ and the Jacobian is diagonalised by Hermite polynomials. The spectral formula complements Hayashi's global convergence theory with a constructive, computable local rate.