Muon g-2 and rare top decays in up-type specific variant axion models

Abstract

The invisible variant axion models (VAM's) offer a very attractive solution for the strong CP problem without the domain wall problem. We consider the up-type specific variant axion models and examine their compatibility with the muon $g-2$ anomaly and the constraints from lepton flavor universality, several flavor observables, and top quark measurements. We find that the combined $\chi^2$ fit favors the parameters $m_A\sim 15$ GeV and $\tan\beta \sim 40$, the same as the type-X 2HDM. Moreover, we find that there are no conflict with any flavor observables as long as the mixing angle $\rho_u$ is sufficiently small. In particular, a small nonzero mixing angle $\rho_u \sim \pi/100$ is slightly favored by the observed $B_s \to \mu\mu$ branching ratio. The up-specific VAM predicts the flavor-violating top rare decay $t\to uA$ followed by $A \to \tau\tau$, which would provide a smoking gun signature at the LHC. We show that current searches of $A$ already impose some constraints on the parameter space but are not sensitive to the most interesting light $m_A$ region. We propose an efficient search strategy that employs di-tau tagging using jet substructure information, and demonstrate that it can enhance the sensitivity on $BR(t \to uA)$, especially in the light $m_A$ region. This model also predicts the flavor-violating decay of heavy Higgs bosons, such as $H \to t u$, that would suppress the $H \to \tau\tau/\mu\mu$ decays. We also examine the up-specific VAM with the muon-specific lepton sector and the down-type specific VAM's as interesting alternative scenarios.