Spin-orbit physics of j=1/2 Mott insulators on the triangular lattice
Abstract
The Heisenberg-Kitaev (HK) model on the triangular lattice is conceptually interesting for its interplay of geometric and exchange frustration. HK models are also thought to capture the essential physics of the spin-orbital entanglement in effective j=1/2 Mott insulators studied in the context of various 5d transition metal oxides. Here we argue that the recently synthesized Ba3IrTi2O9 is a prime candidate for a microscopic realization of the triangular HK model. We establish that an infinitesimal Kitaev exchange destabilizes the 120 order of the quantum Heisenberg model and results in the formation of an extended Z2-vortex crystal phase in the parameter regime most likely relevant to the real material. Using a combination of analytical and numerical techniques we map out the entire phase diagram of the model, which further includes various ordered phases as well as an extended nematic phase around the antiferromagnetic Kitaev point.