Model-free Rank Aggregation in the Presence of Rater Heterogeneity: A Maximum Score Approach

Abstract

This paper investigates the rank aggregation problem through the lens of multi-way comparison data derived from rater scores. Departing from traditional parametric frameworks, such as the Bradley-Terry and Plackett-Luce models, we propose a model-free method that accommodates highly heterogeneous preference distributions across raters and encompasses weak stochastic transitivity in pairwise comparisons as a special case. We establish the theoretical foundations of the proposed estimator by proving its consistency, demonstrating that the proportion of discordant pairs (Kendall tau) converges to zero in probability as the number of raters diverges. Furthermore, we derive upper and lower bounds for a performance metric based on Kendall's tau. In certain asymptotic regimes, these bounds coincide up to logarithmic factors, so the estimator is nearly minimax optimal. These results are obtained by analyzing the convergence behavior of a U-empirical process; the novel technical results developed for this analysis may be of independent theoretical interest. The practical utility of our method is validated through extensive simulations and applications to sports player rankings and survey preference aggregation.