Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors

Zhenghua Bao


Abstract
Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/Continuum-AI-Corp/spoken-multihop-rag.
Anthology ID:
2026.emnlp-main.425
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
9742–9756
Language:
URL:
https://aclanthology.org/2026.emnlp-main.425/
DOI:
Bibkey:
Cite (ACL):
Zhenghua Bao. 2026. Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 9742–9756, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors (Bao, EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.425.pdf