Twisted relative trace formulae with a view towards unitary groups

Abstract

We introduce a twisted relative trace formula which simultaneously generalizes the twisted trace formula of Langlands et.al. (in the quadratic case) and the relative trace formula of Jacquet and Lai. Certain matching statements relating this twisted relative trace formula to a relative trace formula are also proven (including the relevant undamental lemma in the "biquadratic case"). Using recent work of Jacquet, Lapid and their collaborators and the Rankin-Selberg integral representation of the Asai $L$-function (obtained by Flicker using the theory of Jacquet, Piatetskii-Shapiro, and Shalika), we give the following application: Let $E/F$ be a totally real quadratic extension with $\langle \sigma \rangle=\mathrm{Gal}(E/F)$, let $U^{\sigma}$ be a quasi-split unitary group with respect to a CM extension $M/F$, and let $U:=\mathrm{Res}_{E/F}U^{\sigma}$. Under suitable local hypotheses, we show that a cuspidal cohomological automorphic representation $\pi$ of $U$ whose Asai $L$-function has a pole at the edge of the critical strip is nearly equivalent to a cuspidal cohomological automorphic representation $\pi'$ of $U$ that is $U^{\sigma}$-distinguished in the sense that there is a form in the space of $\pi'$ admitting a nonzero period over $U^{\sigma}$. This provides cohomologically nontrivial cycles of middle dimension on unitary Shimura varieties analogous to those on Hilbert modular surfaces studied by Harder, Langlands, and Rapoport.