Rankin-Selberg convolutions for $\mathrm{GL}(n)\times \mathrm{GL}(n)$ and $\mathrm{GL}(n)\times \mathrm{GL}(n-1)$ for principal series representations

Abstract

Let $\mathsf k$ be a local field. Let $I_\nu$ and $I_{\nu'}$ be smooth principal series representations of $\mathrm{GL}_n(\mathsf k)$ and $\mathrm{GL}_{n-1}(\mathsf k)$ respectively. The Rankin-Selberg integrals yield a continuous bilinear map $I_\nu\times I_{\nu'}\rightarrow \mathbb C$ with a certain invariance property. We study integrals over a certain open orbit that also yield a continuous bilinear map $I_\nu\times I_{\nu'}\rightarrow \mathbb C$ with the same invariance property, and show that these integrals equal the Rankin-Selberg integrals up to an explicit constant. Similar results are also obtained for Rankin-Selberg integrals for $\mathrm{GL}_n(\mathsf k)\times \mathrm{GL}_n(\mathsf k)$.