Momentum-Resolved Fingerprint of Mottness in Layer-Dimerized Nb$_3$Br$_8$

Abstract

In a well-ordered crystalline solid, insulating behaviour can arise from two mechanisms: electrons can either scatter off a periodic potential, thus forming band gaps that can lead to a band insulator, or they localize due to strong interactions, resulting in a Mott insulator. For an even number of electrons per unit cell, either band- or Mott-insulators can theoretically occur. However, unambiguously identifying an unconventional Mott-insulator with an even number of electrons experimentally has remained a longstanding challenge due to the lack of a momentum-resolved fingerprint. This challenge has recently become pressing for the layer dimerized van der Waals compound Nb$_3$Br$_8$, which exhibits a puzzling magnetic field-free diode effect when used as a weak link in Josephson junctions, but has previously been considered to be a band-insulator. In this work, we present a unique momentum-resolved signature of a Mott-insulating phase in the spectral function of Nb$_3$Br$_8$: the top of the highest occupied band along the out-of-plane dimerization direction $k_z$ has a momentum space separation of $\Delta k_z=2\pi/d$, whereas the valence band maximum of a band insulator would be separated by less than $\Delta k_z=\pi/d$, where $d$ is the average spacing between the layers. As the strong electron correlations inherent in Mott insulators can lead to unconventional superconductivity, identifying Nb$_3$Br$_8$ as an unconventional Mott-insulator is crucial for understanding its apparent time-reversal symmetry breaking Josephson diode effect. Moreover, the momentum-resolved signature employed here could be used to detect quantum phase transition between band- and Mott-insulating phases in van der Waals heterostructures, where interlayer interactions and correlations can be easily tuned to drive such transition.