Lepton flavor violating Higgs boson decay $h\to\mu\tau$ at the ILC

Abstract

We study the possible reach of the proposed International Linear Collider (ILC) in exploring the lepton flavor violating (LFV) Higgs boson decay $h\to\mu\tau$. Two generic types of models are investigated, both involving an extended scalar sector. For the first type, the rest of the scalars are heavy and beyond the reach of ILC but the LFV decay occurs through a tiny admixture of the Standard Model (SM) doublet with the heavy degrees of freedom. In the second class, which is more constrained from the existing data, the SM Higgs boson does not have any LFV decay but there are other scalars degenerate with it that gives rise to the LFV signal. We show that ILC can pin down the branching fraction of $h\to\mu\tau$, and hence the effective LFV Yukawa coupling, to a very small value, due to the fact that there are signal channels with unlike flavor leptons but no missing energy. It turns out that the low-energy options of the ILC, namely, $\sqrt{s}=250$ or 500 GeV are better to investigate such channels, and the option of beam polarization helps too. At least an order of magnitude improvement is envisaged over the existing limits.