ContextRouter: Learning When to Use Context for Efficient LLM Machine Translation

Ritvik Choudhary, Rem Hida, Toshiyuki Sekiya


Abstract
LLM-based machine translation systems can now do more than translate a source segment in isolation. At test time, they can translate directly, prepend previous context, or refine a context-aware draft. However, these interventions are not uniformly beneficial. Context can resolve discourse-sensitive choices, but irrelevant context and extra refinement increases cost and may perturb adequate translations. We introduce ContextRouter, a lightweight policy router that learns which intervention to apply for each source segment. During training, we first generate candidates for all contextual interventions and score them with a routing objective combining overall translation quality with incurred token costs. We then train our ContextRouter with group-relative supervision, so it learns within-example preferences rather than absolute global action scores. At inference, our router uses only test-time-available signals from the source, previous source-side context, and a no-context draft to determine and execute only the selected context intervention. On a filtered context-aware subset of TED, QED, and OpenSubtitles covering multiple languages and domains, our ContextRouter improves the quality-cost tradeoff over fixed no-context, always-context, always-refinement, QE-reranking, and score-regression policies, and human evaluation confirms that annotators prefer its chosen interventions. These results support treating context and refinement actions in LLM-MT not as always present defaults but conditional test-time interventions instead.
Anthology ID:
2026.emnlp-main.1635
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
37164–37183
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1635/
DOI:
Bibkey:
Cite (ACL):
Ritvik Choudhary, Rem Hida, and Toshiyuki Sekiya. 2026. ContextRouter: Learning When to Use Context for Efficient LLM Machine Translation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 37164–37183, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ContextRouter: Learning When to Use Context for Efficient LLM Machine Translation (Choudhary et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1635.pdf