HarmoLoRA: Commit-Stage Update Admission in Low-Rank Adaptation for Continual Speech Recognition

Ye-Eun Ko, Ji-Eun Choi, Dong-Hyun Kim, Joon-Hyuk Chang


Abstract
Replay-free continual automatic speech recognition (ASR) requires adapting to new speech domains while maintaining a single persistent model without old data, task identifiers, or inference-time routing. We identify commit-stage update admission as a distinct post-training problem for continual low-rank adaptation (LoRA): after a task-wise LoRA update has been trained, the system must decide which part of that update should become persistent in the deployed model. This decision is especially important in ASR, where domains share phonetic and transcription structure but still differ in speakers, channels, noise, speaking style, and lexical coverage. We propose HarmoLoRA, a rank-constrained commit operator that decomposes the trained current-task update with respect to previously committed directions, constructs a rank-closed residual commit path, and uses a lexicographic constrained rule to choose the smallest residual admission coefficient among near-best realized candidates. The selected update is committed to one merged ASR checkpoint, requiring no replay, task identifiers, routing, gating, or multi-branch selection at inference. Across the reported ASR orders, HarmoLoRA improves recognition performance and reduces forgetting relative to reproduced replay-free and LoRA-based continual adaptation baselines.
Anthology ID:
2026.emnlp-main.2631
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
59824–59841
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2631/
DOI:
Bibkey:
Cite (ACL):
Ye-Eun Ko, Ji-Eun Choi, Dong-Hyun Kim, and Joon-Hyuk Chang. 2026. HarmoLoRA: Commit-Stage Update Admission in Low-Rank Adaptation for Continual Speech Recognition. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 59824–59841, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
HarmoLoRA: Commit-Stage Update Admission in Low-Rank Adaptation for Continual Speech Recognition (Ko et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2631.pdf