Phases of triangular lattice antiferromagnet near saturation

Abstract

We consider 2D Heisenberg antiferromagnets on a triangular lattice with spatially anisotropic interactions in a high magnetic field close to the saturation. We show that this system possess rich phase diagram in field/anisotropy plane due to competition between classical and quantum orders: an incommensurate non-coplanar spiral state, which is favored classically, and a commensurate co-planar state, which is stabilized by quantum fluctuations. We show that the transformation between these two states is highly non-trivial and involves two intermediate phases -- the phase with co-planar incommensurate spin order and the one with non-coplanar double-${\bf Q}$ spiral order. The transition between the two co-planar states is of commensurate- incommensurate type, not accompanied by softening of spin-wave excitations. We show that a different sequence of transitions holds in triangular antiferromagnets with exchange anisotropy, such as Ba3CoSb2O9.