Topological quantum phase transitions driven by a displacement field in the twisted MoTe2 bilayers

Abstract

We study twisted bilayer MoTe$_2$ systems at fractional fillings of the lowest hole band under an applied out-of-plane displacement field. By employing exact diagonalization in finite-size systems, we systematically map out the ground state quantum phase diagram for two filling fractions, $\nu=1/3$ and $2/3$, and provide a detailed characterization of each phase. We identify the phase transition between a fractional Chern insulator (FCI) and a layer-polarized charge density wave (CDW) at a filling fraction of $\nu=1/3$, denoted as CDW-$1$. Additionally, we demonstrate that the competition between the displacement field and twist angle leads to another phase transition from a layer-polarized CDW-$1$ to a layer-hybridized CDW-$2$, identified as a first-order phase transition. Furthermore, at $\nu=2/3$ filling of the lowest hole band, we observe that the FCI remains stable against the displacement field until it approaches proximity to a transition in single-particle band topology at a smaller twist angle.