An Objective Intelligibility Metric Evaluation on Spanish Speech

Abstract

Objective intelligibility metrics (OIMs) enable fast and low-cost evaluation of speech intelligibility and are widely used in speech technology assessment. In this study, we evaluate five reference-based OIMs (STOI, ESTOI, STGI, HASPI, and SIIB) and two deep learning-based no-reference metrics (MOSA-Net+ and W2V-SIP) on SpInt, a new Spanish speech intelligibility dataset. Our results show that reference-based OIMs consistently outperform modern data-driven no-reference approaches, which degrade notably under training-test acoustic mismatches such as language mismatch. This effect is particularly relevant in our scenario, as none of the evaluated metrics were exposed to Spanish speech data during development. Consequently, to foster research on more robust and generalizable no-reference OIMs, SpInt is released publicly.