Bessel convolutions on matrix cones

Abstract

In this paper we introduce probability-preserving convolution algebras on cones of positive semidefinite matrices over one of the division algebras $\b F = \b R, \b C$ or $\b H$ which interpolate the convolution algebras of radial bounded Borel measures on a matrix space $M_{p,q}(\b F)$ with $p\geq q$. Radiality in this context means invariance under the action of the unitary group $U_p(\b F)$ from the left. We obtain a continuous series of commutative hypergroups whose characters are given by Bessel functions of matrix argument. Our results generalize well-known structures in the rank one case, namely the Bessel-Kingman hypergroups on the positive real line, to a higher rank setting. In a second part of the paper, we study structures depending only on the matrix spectra. Under the mapping $r\mapsto \text{spec}(r)$, the convolutions on the underlying matrix cone induce a continuous series of hypergroup convolutions on a Weyl chamber of type $B_q$. The characters are now Dunkl-type Bessel functions. These convolution algebras on the Weyl chamber naturally extend the harmonic analysis for Cartan motion groups associated with the Grassmann manifolds $U(p,q)/(U_p\times U_q)$ over $\b F$.