Perceptually Better, Semantically Worse: Measuring Speech Enhancement Impact on LLM-Based Voice Systems

Randy Frans Fela, Pejman Mowlaee


Abstract
Speech enhancement (SE) is commonly applied as a preprocessing step in spoken AI pipelines under the assumption that better audio quality improves downstream task performance. Whether SE-induced distortions propagate to downstream LLM task performance remains an open question. We introduce Output Divergence Rate (ODR), which measures how often SE changes an LLM’s intent classification relative to clean speech, and benchmark five conditions on 2,974 SLURP clips using Whisper large-v3 and wav2vec2-large cascades. Every condition produces ODR significantly above zero (p < 0.001, binomial test). MetricGAN+ more than doubles ODR versus unenhanced noisy speech (0.318 vs. 0.135) despite improving PESQ, and unmitigated echo reaches an ODR of 0.836 through speaker substitution, a failure WER cannot capture. Audio quality metrics range from near-zero to moderate correlation with ODR (SQUIM-MOS 𝜌\=\-0.068, PESQ 𝜌\=\-0.467). The MetricGAN+ and echo results replicate across ASR architectures, indicating that standard audio quality metrics are insufficient for LLM pipeline quality.
Anthology ID:
2026.emnlp-industry.112
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1913–1924
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.112/
DOI:
Bibkey:
Cite (ACL):
Randy Frans Fela and Pejman Mowlaee. 2026. Perceptually Better, Semantically Worse: Measuring Speech Enhancement Impact on LLM-Based Voice Systems. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1913–1924, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Perceptually Better, Semantically Worse: Measuring Speech Enhancement Impact on LLM-Based Voice Systems (Fela & Mowlaee, EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.112.pdf