Relative trace formulae toward Bessel and Fourier-Jacobi periods of unitary groups

Abstract

We propose a relative trace formula approach and state the corresponding fundamental lemma toward the global restriction problem involving Bessel or Fourier-Jacobi periods of unitary groups $\mathrm{U}_n\times\mathrm{U}_m$, extending the work of Jacquet-Rallis for $m=n-1$ (which is a Bessel period). In particular, when $m=0$, we recover a relative trace formula proposed by Flicker concerning Kloosterman/Fourier integrals on quasi-split unitary groups. As evidence for our approach, we prove the fundamental lemma for $\mathrm{U}_n\times\mathrm{U}_n$ in positive characteristics.