Unfolding quantum entanglement from $h\to ZZ^*\to jj\ell\ell$ at a muon collider

Abstract

We explore the potential to study quantum entanglement through Bell-type inequalities in Higgs boson decays at a future muon collider. Our analysis focuses on the channel $\mu^+ \mu^- \to \nu \bar{\nu} h \to \nu \bar{\nu} ZZ^*$, with one $Z$ decaying to charged leptons and the other decaying hadronically into jets. We study the violation of the CGLMP inequality using the optimal Bell operator for the bipartite qutrit system from $h \to ZZ^*$. The entanglement measure $\mathcal{I}_3$ is constructed from spin-correlated angular observables of the $Z$ decay products. An unfolding method on the angular variables is applied to correct for hadronization and detector effects, recovering the advantage of the hadronic mode with higher event yield and reduced uncertainty. The study is performed at 1, 3, and 10 TeV centre-of-mass energies, assuming 10 ab$^{-1}$ integrated luminosity for each case. At 1 TeV, we use a boosted decision tree for signal isolation, while at higher energies, simple cut-based analyses are sufficient. We find clear Bell inequality violation with the expected values $\mathcal{I}_3 = 2.625 \pm 0.012$, $2.623 \pm 0.004$, and $2.582 \pm 0.010$ for the 1, 3, and 10 TeV machines, respectively. Overall, a strong level of entanglement close to the maximum achievable value of 2.9149 for a two-qutrit system can be measured with very small uncertainties due to the large event yield in the hadronic mode.