UMelCodec: An Ultra-Low-Bitrate and Lightweight Mel-Domain Neural Speech Codec
Hui-Peng Du, Yang Ai, Zhen-Hua Ling
Abstract
Ultra-low-bitrate speech coding is crucial for bandwidth-constrained communication and compact storage, but preserving intelligibility, naturalness, and fine-grained acoustic details at bitrates of only a few hundred bps remains challenging. Existing purely acoustic codecs generally operate at relatively high bitrates, whereas incorporating semantic representations can reduce the bitrate, but at the cost of increased complexity. Therefore, we propose UMelCodec, a purely acoustic neural speech codec operating in the mel domain at an ultra-low bitrate of 325 bps with a lightweight architecture. UMelCodec consists of two stages. In the first mel-domain coding stage, alias-free Snake-ConvNeXt (AF-SC) encoder-decoder backbones model temporal and harmonic structures, a factorized single-codebook vector quantizer (VQ) enables ultra-low-bitrate quantization, and a multi-scale mel-spectrogram discriminator (MSMD) improves mel-spectrogram reconstruction. In the second waveform synthesis stage, a lightweight modified discrete cosine transform (MDCT)-based neural vocoder predicts the MDCT spectrum from the reconstructed mel spectrogram and recovers the waveform through inverse MDCT. Experiments on LibriTTS dataset demonstrate that UMelCodec outperforms strong baselines across most objective reconstruction metrics while maintaining competitive subjective naturalness. In terms of model complexity and inference efficiency, UMelCodec has only 19.67 M parameters, requires 1.93 G MACs, and supports real-time inference on both GPU and CPU platforms.
Create a lesson
Related papers
Blind tests of a simple-body model of submarine target strength on open benchmarks
Sunil Tyagi
Joint Language Modeling for In-the-Wild Speech-Scene Generation with Distinct Token Representations
Semin Kim, Ju Yeon Kang, Sunghwan Ahn et al.
Sex Differences in the Detection of Parkinson's Disease from Speech
Hiba Akhaddar, Peter Plantinga, Mirco Ravanelli et al.
Location Matters: Layer-wise Prompt Conditioning for Voice--Style Disentanglement in Text-to-Speech
David Lindevelt, Suzan Verberne, Joost Broekens
21Codec: Decoder-side Speaker Control for Text-to-Speech
David Lindevelt, Suzan Verberne, Joost Broekens
Beyond Intelligibility: Fine-Grained Perceptual Assessment of Dysarthric Speech with Auditory Large Language Models
Haotian Wei, Can Long, Wen Xian Jim et al.