Skip to content

MOUSSE: Multilingual Overlapped Speech Separation with Predictive Uncertainty Estimation

Su Yeon Shim, Yejin Jeon, David Ifeoluwa Adelani, Young-Joo Suh

cs.SDarXiv:2611.19706

Abstract

Recent generative speech separation methods have shown promising performance, yet mixtures of overlapping speech in different languages remain underexplored. To this end, we first curate MuSSeD, an English-Korean benchmark comprising two- to four- speaker mixtures across varying overlap ratios. We further propose MOUSSE, a flow matching framework that incorporates boundary-aware training and frame-wise predictive uncertainty through a permutation-invariant Gaussian negative log-likelihood. The predicted scale is further used to adapt classifier-free guidance during sampling. MOUSSE achieves 13.18 dB SI-SDRi and 0.175 ESTOIi, outperforming discriminative and flow matching-based separation baselines, with a 1.58 dB SI-SDRi improvement over the strongest model.

Create a lesson