Testing Realistic Quark Mass Matrices in the Custodial Randall-Sundrum Model with Flavor Changing Top Decays

Abstract

We study quark mass matrices in the Randall-Sundrum (RS) model with bulk symmetry $SU(2)_L \times SU(2)_R \times U(1)_{B-L}$. The Yukawa couplings are assumed to be within an order of magnitude of each other, and perturbative. We find that quark mass matrices of the symmetrical form proposed by Koide \textit{et. al.} [Y. Koide, H. Nishiura, K. Matsuda, T. Kikuchi and T. Fukuyama, Phys. Rev. D {\bf 66}, 093006 (2002)] can be accommodated in the RS framework with the assumption of hierarchyless Yukawa couplings, but not the hermitian Fritzsch-type mass matrices. General asymmetrical mass matrices are also found which fit well simultaneously with the quark masses and the Cabibbo-Kobayashi-Maskawa matrix. Both left-handed (LH) and right-handed (RH) quark rotation matrices are obtained that allow analysis of flavour changing decay of both LH and RH top quarks. At a warped down scale of 1.65 TeV, the total branching ratio of $t \ra Z$ + jets can be as high as $\sim 5 \times 10^{-6}$ for symmetrical mass matrices and $\sim 2 \times 10^{-5}$ for asymmetrical ones. This level of signal is within reach of the LHC.