Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning

Shunta Asano, Jeonghun Baek, Toshihiko Yamasaki


Abstract
Recent studies have shown that code-switching data (CSD), in which multiple languages are mixed within the same context, can improve cross-lingual transfer and multilingual alignment in large language models (LLMs). Recent studies on synthetic code-switching for LLMs have primarily focused on bilingual transfer between English and a target language, leaving instruction-tuning settings that jointly mix three or more languages within each example largely unexplored. In this work, we investigate multilingual code-switching instruction tuning across four languages: English, Japanese, Korean, and Chinese. We evaluate multilingual understanding on Belebele and MMLU. Our experiments show that simple sentence-level multilingual CSD consistently improves the four-language average accuracy across all bilingual, trilingual, and quadrilingual settings for Gemma-2-2B, indicating that multilingual code-switching can be effective beyond bilingual transfer settings.
Anthology ID:
2026.findings-aacl.2
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
23–34
Language:
URL:
https://aclanthology.org/2026.findings-aacl.2/
DOI:
Bibkey:
Cite (ACL):
Shunta Asano, Jeonghun Baek, and Toshihiko Yamasaki. 2026. Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 23–34, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning (Asano et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.2.pdf