Limits of Bessel functions for root systems as the rank tends to infinity

Abstract

We study the asymptotic behaviour of Bessel functions associated of root systems of type $A_{n-1}$ and type $B_n$ with positive multiplicities as the rank $n$ tends to infinity. In both cases, we characterize the possible limit functions and the Vershik-Kerov type sequences of spectral parameters for which such limits exist. In the type $A$ case, this gives a new and very natural approach to recent results by Assiotis and Najnudel in the context of $\beta$-ensembles in random matrix theory. These results generalize known facts about the approximation of the (positive-definite) Olshanski spherical functions of the space of infinite-dimensional Hermitian matrices over $\mathbb F = \mathbb R, \mathbb C, \mathbb H$ (with the action of the associated infinite unitary group) by spherical functions of finite-dimensional spaces of Hermitian matrices. In the type B case, our results include asymptotic results for the spherical functions associated with the Cartan motion groups of non-compact Grassmannians as the rank goes to infinity, and a classification of the Olshanski spherical functions of the associated inductive limits.