Localizing Cross-Lingual Generalization in Parameter Space

Yeoreum Lee, Jaeseong Lee, Seung-Won Hwang


Abstract
Cross-lingual transfer is uneven across tasks, succeeding for natural language understanding but often failing for mathematical reasoning. We argue that this asymmetry is governed by the geometry of parameter space. Specifically, cross-lingual generalization is determined by whether English and target-language LoRA adapters share a low-error basin. To test this, we use linear mode connectivity (LMC) between English and target-language adapters, evaluating on three tasks. We find that NLI and QA model-language pairs are mostly linearly connected, whereas mathematical reasoning (Math) pairs are not, indicating that basin geometry is task-dependent. Such disconnection in Math is “local” rather than global. A small target-language task sample, applied only to the English adapter’s language encoding or semantic composition layers, is enough to repair it. Cross-lingual generalization is also factorizable into language-specific encoding and language-agnostic composition, each supplied separately from a cheaper signal. English task data routed to the composition layers and an unlabeled target-language corpus routed to the language encoding layers each alleviate it on their own for nearly every pair. Composition is the more influential of these, and it needs no target-language data. Together, these results suggest that cross-lingual generalization decomposes into language-specific and language-agnostic components in parameter space. This decomposition offers a practical route for low-resource cross-lingual adaptation.
Anthology ID:
2026.emnlp-main.2625
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
59673–59692
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2625/
DOI:
Bibkey:
Cite (ACL):
Yeoreum Lee, Jaeseong Lee, and Seung-Won Hwang. 2026. Localizing Cross-Lingual Generalization in Parameter Space. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 59673–59692, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Localizing Cross-Lingual Generalization in Parameter Space (Lee et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2625.pdf