Tube domains and restrictions of minimal representations

Abstract

In this paper we study the restrictions of the minimal representation in the analytic continuation of the scalar holomorphic discrete series from $Sp(n,\mathbb{R})$ to $GL(n,\mathbb{R})$, and from SU(n,n) to $GL(n,\mathbb{C})$ respectively. We work with the realisations of the representation spaces as $L^2$-spaces on the boundary orbits of rank one of the corresponding cones, and give explicit integral operators that play the role of the intertwining operators for the decomposition. We prove inversion formulas for dense subspaces and use them to prove the Plancherel theorem for the respective decomposition. The Plancherel measure turns out to be absolutely continuous with respect to the Lebesgue measure in both cases.