Ground-state phase diagram of the Kitaev-Heisenberg model on a kagome lattice

Abstract

The Kitaev-Heisenberg model on the honeycomb lattice has been studied for the purpose of finding exotic states such as quantum spin liquid and topological orders. On the kagome lattice, in spite of a spin-liquid ground state in the Heisenberg model, the stability of the spin-liquid state has hardly been studied in the presence of the Kitaev interaction. Therefore, we investigate the ground state of the classical and quantum spin systems of the kagome Kitaev-Heisenberg model. In the classical system, we obtain an exact phase diagram that has an eight-fold degenerated canted ferromagnetic phase and a subextensive degenerated Kitaev antiferromagnetic phase. In the quantum system, using the Lanczos-type exact diagnalization and cluster mean-field methods, we obtain two quantum spin-liquid phases, an eight-fold degenerated canted ferromagnetic phase similar to the classical spin system, and an eight-fold degenerated $\bf q=0$ $120^\circ$ ordered phase induced by quantum fluctuation. These results may provide a crucial clue to recently observed magnetic structures of the rare-earth-based kagome lattice compounds $A_2$RE$_3$Sb$_3$O$_{14}$ ($A$ = Mg, Zn; RE = Pr, Nd, Gd, Tb, Dy, Ho, Er, Yb).