Stabilization and modulation of the topological magnetic phase with a $Z_2$-vortex lattice in the Kitaev-Heisenberg honeycomb model: The key role of the third-nearest-neighbor interaction

Abstract

The topologically nontrivial magnetic phase with a $Z_2$-vortex (Z$_2$V) lattice is investigated by simulation in the Kitaev-Heisenberg honeycomb model expanded by considering the second- and third-nearest-neighbor Heisenberg interactions ($J_{H2}$ and $J_{H3}$). On the parameter region of the Z$_2$V phase, a gradual modulation of vortex density is observed, together with a transition from single-Z$_2$V to triple-Z$_2$V state driven by the variation of frustration. Additionally, $Z_2$ vortices are arranged in different manners on the whole honeycomb structure for these two types of Z$_2$V states. Moreover, topologically equivalent states are revealed to exist in single-Z$_2$V dominant and triple-Z$_2$V dominant styles on different parameter points, which can be controlled to switch between each other without energy consumption. It is worth noting that $J_{H3}$ plays a key role in expanding the Z$_2$V phase, and also in stabilizing the single-Z$_2$V state.