Nucleation beyond Equilibrium: Fronts Control Invasion in Bistable Ecosystems

Abstract

Bistability, the existence of two alternative stable states with distinct basins of attraction, is common across ecology and many other biological, chemical, and physical systems. In spatially extended systems, the invasion of one state by the other proceeds through nucleation, the fluctuation-driven growth of a droplet beyond a critical size. Classical Nucleation Theory (CNT) quantifies this process using the energy landscape, but this description relies on detailed balance, the condition of microscopic reversibility that holds at equilibrium. Ecological dynamics generally violate detailed balance and are described not by a single scalar field but by several coupled, non-conserved abundances--a vector order parameter--for which no general nucleation theory exists. Here we derive such a theory for reaction-diffusion systems with a non-conserved vector order parameter, valid both close to the binodal, where invasion proceeds through propagating fronts, and close to the spinodal, where the metastable state loses stability. Extensive numerical computations of the quasipotential confirm the theory in both regimes. We show that the mathematical structure of CNT survives out of equilibrium, once energetic quantities are replaced by dynamical properties of fronts: the front speed plays the role of the bulk free-energy difference between phases, and the diffusivity that of the surface tension. Applied to the two-species Lotka-Volterra model, an archetypal system of bistable ecological antagonism, our theory shows that strong interspecific competition generates a pronounced depletion region within fronts, where the total abundance falls well below carrying capacity. This vectorial structure, invisible to a scalar description based on species frequency alone, makes invasion exponentially harder as competition strengthens at fixed competitive advantage--a prediction testable in microbial systems.