A variational Monte Carlo approach to the SU(4) spin-orbital model on the triangular lattice

Abstract

Previous investigations have suggested that the simplest spin-orbital model on the simplest frustrated lattice can host a nematic quantum spin-orbital liquid state. Namely, the orbital degeneracy of the SU(4) Kugel-Khomskii (KK) model tends to enhance quantum fluctuations and stabilize a quantum spin-orbital liquid exhibiting stripy features on the triangular lattice, as revealed by the state-of-the-art method of the density matrix renormalization group boosted by Gutzwiller projected wave functions. In this work, using the variational quantum Monte Carlo method, we have studied several spin-orbital liquid states, including a uniform $\pi$ flux state, three stripy states, and a plaquette state, on the $L\times{}L$ torus up to $L=24$. It turns out that one of these stripy states, called the "stripe-II" state, is energetically favored. This ground state breaks the $C_6$ symmetry of the lattice, resulting in a reduced $C_2$ symmetry and doubled unit cells, while preserving the SU(4) spin-orbital rotation symmetry. Such a nematic quantum spin-orbital liquid state can be characterized by a parton Fermi surface (FS) consisting of open orbits in the Brillouin zone, in contrast to the circular FS of the uniform $\pi$-flux state.