Introducing Code-Switched Contexts to Cognitively-Inspired Bilingual Model Training

Zhuojing Huang, Luise Pohlmann, Lisa Beinborn


Abstract
During language acquisition, bilingual children are regularly exposed to code-switched input and use it as a cognitive scaffold to accelerate vocabulary growth and cross-linguistic syntactic mapping. In contrast, computational bilingual models are conventionally pretrained on interleaved monolingual corpora. While introducing synthetic code-switching during pretraining has become a promising strategy to enhance cross-lingual alignment and downstream performance, the structural and developmental parameters governing the success remain poorly understood. In this work, we investigate the efficiency of training with synthetic code-switched data across two typologically distinct language pairs by controlling two key variables: the structural location of code-switches and the dynamic switching rate across training stages. Our results show that training with code-switched data improves cross-lingual alignment for typologically close languages.
Anthology ID:
2026.babylm-main.62
Volume:
Proceedings of the Second BabyLM Workshop
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Jaap Jumelet, Tal Linzen, Aaron Mueller, Suchir Salhan, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox
Venues:
BabyLM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
805–822
Language:
URL:
https://aclanthology.org/2026.babylm-main.62/
DOI:
Bibkey:
Cite (ACL):
Zhuojing Huang, Luise Pohlmann, and Lisa Beinborn. 2026. Introducing Code-Switched Contexts to Cognitively-Inspired Bilingual Model Training. In Proceedings of the Second BabyLM Workshop, pages 805–822, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Introducing Code-Switched Contexts to Cognitively-Inspired Bilingual Model Training (Huang et al., BabyLM 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.babylm-main.62.pdf