Hom schemes for algebraic groups

Abstract

In SGA3, Demazure and Grothendieck showed that if $G$ and $H$ are smooth affine group schemes over a scheme $S$ and $G$ is reductive, then the functor of $S$-homomorphism $G \to H$ is representable. In this paper we extend this result to cover cases in which $G$ is not reductive, with much simpler proofs. Our results apply in particular to parabolics over any base, and they are essentially optimal over a field. We also relate the closed orbits in Hom schemes to Serre's theory of complete reducibility, answer a question of Furter--Kraft, and provide many examples.