Stein Variational Gradient Descent dynamics for highly concentrated kernels

Abstract

Stein Variational Gradient Descent (SVGD) is a widely used in practice algorithm for scalable sampling with deterministic particle updates. We study its behavior in the singular limit where the kernel bandwidth tends to zero. In this regime, we show that the nonlocal SVGD dynamics converge to a local evolution equation that can be formally interpreted as a Wasserstein gradient flow with quadratic mobility. We analyze this singular limit in two settings: integrable kernels and weighted kernels. In the weighted case, the proof is supported by recently established Stein-log-Sobolev inequalities, which provide the necessary functional control. Overall, our results clarify how SVGD collapses from a nonlocal interacting particle system to a local gradient-flow dynamics as the kernel concentrates.