Quantum Phase Transition in the SU(4) Spin-Orbital Model on the Triangular Lattice

Abstract

Motivated by the absence of cooperative Jahn-Teller effect in LiNiO2 and BaVS3, two layered oxides with triangular planes, we study the SU(4) symmetric spin-orbital model on the triangular lattice. Upon reducing the next-nearest neighbour coupling, we show that the system undergoes a quantum phase transition to a liquid phase. A variational approach to this liquid phase shows that simple types of long-range correlations are suppressed, suggesting that it is stable against lattice distortions.