Magnetic phase diagram slightly below the saturation field in the stacked $J_1$-$J_2$ model in the square lattice with the $J_{\text{C}}$ interlayer coupling

Abstract

We study the effect of adding interlayer coupling to the square lattice, $J_1$-$J_2$ Heisenberg model in high external magnetic field. In particular, we consider a cubic lattice formed from stacked $J_1$-$J_2$ layers, with interlayer exchange coupling $J_{\text{C}}$. For the 2-dimensional model ($J_{\text{C}}=0$) it has been shown that a spin-nematic phase appears close to the saturation magnetic field for the parameter range $-0.4 \lesssim J_2/J_1$ and $J_2>0$. We determine the phase diagram for 3-dimensional model at high magnetic field by representing spin flips out of the saturated state as bosons, considering the dilute boson limit and using the Bethe-Salpeter equation to determine the first instability of the saturated paramagnet. Close to the highly frustrated point $J_2/J_1\sim0.5$, we find that the spin-nematic state is stable even for $|J_{\text{C}}/J_1|\sim 1$. For larger values of $J_2/J_1$, interlayer coupling favors a broad, phase-separated region. Further increase of $|J_{\text{C}}|$ stabilizes a collinear antiferromagnet, which is selected via the order-by-disorder mechanism.