DrEM: Dual-Side Robust Ensemble Ranking from Noisy User Preference Predictions in Video Recommendation

Abstract

Industrial video recommendation systems typically adopt a multi-stage architecture. At the ensemble ranking stage, multi-dimensional user preference predictions (pxtrs) from an upstream multi-task model are fused into a unified ranking score to reflect user satisfaction. Since users' true satisfaction is difficult to observe directly, ensemble ranking models commonly use pxtrs both as input features and as a source for constructing proxy preferences. However, as outputs of an upstream prediction model, pxtrs inevitably contain prediction noise, which propagates to downstream learning across two sides. On the supervision side, noisy pxtrs may flip proxy preferences and introduce erroneous gradients. On the feature side, pxtr noise may propagate through model inputs and destabilize ranking scores. Existing ensemble ranking methods typically treat pxtrs as reliable signals and overlook such prediction noise. To address this, we propose DrEM, a dual-side robust ensemble ranking framework. Our DrEM introduces a risk-denoising robust loss that corrects the empirical risk using estimated preference flip probability. Meanwhile, it samples perturbations from the distribution of prediction noise and introduces a preference-preserving ranking consistency regularizer to improve feature-side output stability. Theoretically, we obtain an approximate distribution of the prediction noise and prove that the robust loss remains superior under flip probability estimation error. Extensive offline experiments and large-scale online A/B tests demonstrate the effectiveness and robustness of our DrEM.