Crosslingual Disparities in LLM Performance: Challenges for MT as Mitigation

Rebecca Knowles, Cyril Goutte


Abstract
We examine crosslingual performance disparities in large language models (LLMs) in the context of safety- and regulation-related queries in Canada. We manually build a set of English and French query pairs with gold standard answers and collect LLM-generated answers, which are manually annotated for correctness. We find that LLMs are more likely to produce errors in their answers in French than in English. We investigate a machine translation pipeline, translating the French query, producing an English LLM response, and translating the response back to French. We find that, while it can mitigate some of these performance disparities, additional challenges such as the reliability and language of the cited sources or technical terms greatly impact that mitigation strategy.
Anthology ID:
2026.amta-research.9
Volume:
Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)
Month:
August
Year:
2026
Address:
Québec City, Canada
Editors:
Eleftheria Briakou, Jeremy Gwinnup, Shivali Goel
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
146–158
Language:
URL:
https://aclanthology.org/2026.amta-research.9/
DOI:
Bibkey:
Cite (ACL):
Rebecca Knowles and Cyril Goutte. 2026. Crosslingual Disparities in LLM Performance: Challenges for MT as Mitigation. In Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 146–158, Québec City, Canada. Association for Machine Translation in the Americas.
Cite (Informal):
Crosslingual Disparities in LLM Performance: Challenges for MT as Mitigation (Knowles & Goutte, AMTA 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.amta-research.9.pdf