An Interpretable Acoustic Coordinate System for Japanese Consonant Morae in Dysarthric Speech
Yuma Enomoto, Misa Yamaguchi, Hiroki Nishikawa, Ittetsu Taniguchi, Kazuhide Matsunaga, Narikazu Uzawa, Takao Onoye
Abstract
Dysarthria reduces speech intelligibility, requiring self-directed rehabilitation based on fine-grained feedback to capture gradual articulatory changes. For Japanese speakers, such feedback is naturally provided in mora units due to its distinct moraic timing and consonant-vowel structure. While recent visualization methods have explored distance-based feedback, most are limited to vowels. Extending them to consonants is challenging, as generative embeddings often smooth transient spectral structures. We propose a visualization system for Japanese consonant morae that maps articulatory deviations onto a two-dimensional space defined by temporal and spectral distances from healthy reference speech. Experiments show consistent geometric separation of consonant categories, with Silhouette Coefficients exceeding 0.5 and origin-based ROC-AUC scores above 0.86. Furthermore, evaluations using real dysarthric speech achieved a high ROC-AUC of 0.897 in screening for unclear pronunciations. Perturbation tests confirm orthogonal temporal and spectral shifts, yielding a physically interpretable mapping for rehabilitation.
Create a lesson
Related papers
ZoionNET: Low-Power Animal Intrusion Detection Using an Acoustic Deep-Learning Classifier for a Cellular Remote-Sensing System
Prasanna Vijay, Adarsh Gupta, Vansh Panwar et al.
When Sounds Turn Heads: Recovering Event Timing That Audio LLMs Fail to Say
Jongbhin Woo, Jaewook Jeong, Joon Son Chung
From Attention to Semantics: Detecting and Mitigating Sound Event Hallucinations in Large Audio-Language Models
Tianyi Sun, Minghao Lv, Chunyang Yu et al.
The landscape of AI speech generation and deepfake detection
Simon King
Exploring Speaker-Aware Engagement Modeling for Children with Autism Spectrum Disorder
Kaleen Shrestha, Austin Joel Dympep, Young Suh Hong et al.
Harmonic Circuit: Chord Progression Lies in a Sparse Principal Information Flow in Symbolic Music Models
Ruiji Yu, Gus Xia