Semiparametric robust mixture of experts based on nonparametric maximum likelihood

Abstract

The mixture of experts (MoE) model provides a flexible approach for modeling heterogeneous regression relationships by allowing covariate-dependent mixing through a gating network, but most existing MoE models rely on parametric assumptions for expert error distributions, typically Gaussian, which can lead to inefficiency and sensitivity to outliers or heavy-tailed behavior when misspecified. We propose a semiparametric MoE model in which each expert error distribution is represented as a nonparametric Gaussian scale mixture estimated via nonparametric maximum likelihood, relaxing parametric assumptions within the Gaussian scale-mixture class while preserving the interpretability and structure of the MoE framework. The resulting model adapts to complex error structures, improves robustness under contamination and heavy tails, and remains competitive under well-specified Gaussian settings, providing a practical and theoretically grounded alternative to parametric MoE formulations.