Extremal rays of the embedded subgroup saturation cone

Abstract

We examine the extremal rays of the cone of dominant weights $(\mu, \widehat\mu)$ for groups $G\subseteq \widehat G$ for which there exists $N \gg0$ such that $$ \left(V(N\mu)\otimes V(N\widehat \mu)\right)^G\ne (0). $$ We exhibit formulas for a class of rays ("type I") on any regular face of the cone. These rays are identified thanks to a generalization of Fulton's conjecture, which we prove along the way. We verify that the remaining rays ("type II") on the face are the images of extremal rays for a smaller cone under a certain map, whose formula is given. A procedure is given for finding the rays of the cone not on any regular face. This is a generalization of the work of Belkale and Kiers on extremal rays for the saturated tensor cone; the specialization is given by $\widehat G = G\times G$ with the diagonal embedding of $G$. We include several examples to illustrate the formulas.