Anisotropic field-induced ordering in the triangular-lattice quantum spin liquid NaYbSe$_2$

Abstract

High-quality single crystals of NaYbSe$_2$, which resembles a perfect triangular-lattice antiferromagnet without the intrinsic disorder, are investigated by magnetization and specific heat, as well as the local probe techniques nuclear magnetic resonance (NMR) and electron spin resonance (ESR). The low-field measurements confirm the absence of any spin freezing or long-range magnetic order down to 50~mK, which suggests a quantum spin liquid ground (QSL) state with gapless excitations. The instability of the QSL state is observed upon applying magnetic fields. For the $H\bot c$ direction, a field-induced magnetic phase transition is observed above 2~T from the $C_{\rm p}(T)$ data, agreeing with a clear $\frac{M_s}{3}$ plateau of $M(H)$, which is associated with an up-up-down (uud) spin arrangement. For the $H\|c$ direction, a field-induced transition could be evidenced at a much higher field range (9 - 21~T). The $^{23}$Na NMR measurements provide microscopic evidence for field-induced ordering for both directions. A reentrant behavior of $T_{\rm N}$, originating from the thermal and quantum spin fluctuations, is observed for both directions. The anisotropic exchange interactions $J_{\perp}\simeq$ 4.7~K and $J_z\simeq$2.33~K are extracted from the modified bond-dependent XXZ model for the spin-$\frac{1}{2}$ triangular-lattice antiferromagnet. The absence of magnetic long-range order at zero fields is assigned to the effect of strong bond-frustration, arising from the complex spin-orbit entangled $4f$ ground state. Finally, we derive the highly anisotropic magnetic phase diagram, which is discussed in comparison with the existing theoretical models for spin-$\frac{1}{2}$ triangular-lattice antiferromagnets.