Degeneracy loci, virtual cycles and nested Hilbert schemes I

Abstract

Given a map of vector bundles on a smooth variety, consider the deepest degeneracy locus where its rank is smallest. We show it carries a natural perfect obstruction theory whose virtual cycle can be calculated by the Thom-Porteous formula. We show nested Hilbert schemes of points on surfaces can be expressed as degeneracy loci. We show how to modify the resulting obstruction theories to recover the virtual cycles of Vafa-Witten and reduced local DT theories. The result computes some Vafa-Witten invariants in terms of Carlsson-Okounkov operators. This proves and extends a conjecture of Gholampour-Sheshmani-Yau and generalises a vanishing result of Carlsson-Okounkov.