A Bending in the Size-mass Relation of Star-forming Galaxies across $0.5 < z < 6.0$ at a Critical Stellar Mass of $10^{10}M_\odot$ Revealed by JWST

Abstract

We investigate the rest-frame optical size-stellar mass relation of galaxies at $0.5<z<6.0$ using deep JWST/NIRCam and MIRI imaging from the PRIMER survey. We find that star-forming galaxies (SFGs) exhibit a broken power-law relation at all redshifts, with a nearly constant pivot mass ($M_{\rm p}$) of $\sim 10^{10} M_\odot$, and a slope flattening above $M_{\rm p}$. This highlights the prevalence of a population of compact, massive SFGs that was underrepresented in previous studies. The size distribution of quiescent galaxies (QGs) is well described by a mixture power-law model, with a pivot mass that increases from $M_{\rm p} \sim 10^{10.0} M_\odot$ at $z =0.75$ to $M_{\rm p} \sim 10^{10.5} M_\odot$ at $z = 2.6$, suggesting that the minimum halo mass required to quench high-mass galaxies increases with redshift. The bending in the size-mass relation of SFGs supports two distinct size growth modes. At $M_{\star} < M_{\rm p}$, size growth is closely coupled to halo growth, while at $M_{\star} > M_{\rm p}$, an increasing fraction of SFGs decouple from halo growth and become compact, likely associated with rapid bulge (and black hole) growth in $M_{\rm h} \gtrsim 10^{12} M_{\odot}$ halos. These compact SFGs are promising progenitors of massive QGs, as evidenced by their similar masses, surface brightness profiles, and morphologies. Their high number densities can account for the observed buildup of massive QGs at $z > 2$, suggesting that the compaction pathway, rather than major mergers of extended SFGs, dominates the formation of high-z massive QGs.