Correlation-driven topological phase transition from in-plane magnetized quantum anomalous Hall to Mott insulating phase in monolayer transition metal trichlorides

Abstract

Based on density functional theory (DFT) calculations, we predict that a monolayer of OsCl$_3$---a layered material whose interlayer coupling is weaker than in graphite---possesses a quantum anomalous Hall (QAH) insulating phase generated by the combination of honeycomb lattice of osmium atoms, their strong spin-orbit coupling (SOC) and ferromagnetic ground state with {\em in-plane} easy-axis. The band gap opened by SOC is \mbox{$E_g \simeq 67$ meV} (or \mbox{$\simeq 191$ meV} if the easy-axis can be tilted out of the plane by an external electric field), and the estimated Curie temperature of such {\em anisotropic planar rotator} ferromagnet is $T_\mathrm{C} \lesssim 350$ K. The Chern number $\mathcal{C}=-1$, generated by the manifold of Os $t_{2g}$ bands crossing the Fermi energy, signifies the presence of a single chiral edge state in nanoribbons of finite width, where we further show that edge states are spatially narrower for zigzag than armchair edges and investigate edge-state transport in the presence of vacancies at Os sites. Since $5d$ electrons of Os exhibit {\em both} strong SOC and moderate correlation effects, we employ DFT+U calculations to show how increasing on-site Coulomb repulsion $U$: gradually reduces $E_g$ while maintaining $\mathcal{C} = -1$ for $0 < U < U_c$; leads to metallic phase with $E_g = 0$ at $U_c$; and opens the gap of topologically trivial Mott insulating phase with $\mathcal{C}=0$ for $U > U_c$.