Rank complement of rational Dyck paths and conjugation of $(m,n)$-core partitions

Abstract

Given a coprime pair $(m,n)$ of positive integers, rational Catalan numbers $\frac{1}{m+n} \binom{m+n}{m,n}$ counts two combinatorial objects:rational $(m,n)$-Dyck paths are lattice paths in the $m\times n$ rectangle that never go below the diagonal; $(m,n)$-cores are partitions with no hook length equal to $m$ or $n$.Anderson established a bijection between $(m,n)$-Dyck paths and $(m,n)$-cores. We define a new transformation, called rank complement, on rational Dyck paths. We show that rank complement corresponds to conjugation of $(m,n)$-cores under Anderson's bijection. This leads to: i) a new approach to characterizing $n$-cores; ii) a simple approach for counting the number of self-conjugate $(m,n)$-cores; iii) a proof of the equivalence of two conjectured combinatorial sum formulas, one over rational $(m,n)$-Dyck paths and the other over $(m,n)$-cores, for rational Catalan polynomials.