Does Typological Distance Affect Cross-lingual Transfer?

Pengfei Ren


Abstract
The Contrastive-Analysis Hypothesis (CAH) proposes that similarities between a learner’s first language (L1) and second language (L2) can facilitate L2 learning, while differences can make it more difficult. We ask a related computational question: under a limited, developmentally motivated data budget, does a coarse near/far relation between L1 and L2 predict grammatical performance when a language model is trained first on L1 and then on L2? We train GPT-2 Small models from scratch on English, Dutch, or Chinese with a fixed Qwen2.5 tokenizer, and continue training them on the corresponding L2 targets. Across five seeds, we compare zero-shot starting scores, terminal L2 scores, Transfer Increment (TI; terminal minus zero-shot), and Transfer Benefit (TB; terminal minus a same-budget target-only baseline). For English, NL→EN and ZH→EN have nearly identical TI. For Dutch, the near-L1 condition has a larger TI, but most of this difference comes from the zero-shot starting scores, while the terminal scores differ by only 0.9 percentage points. Headroom-normalized TI gives the same target-dependent picture. Across the tested pairs, the near/far contrast therefore does not produce a uniform, target-independent advantage for the near L1.
Anthology ID:
2026.babylm-main.6
Volume:
Proceedings of the Second BabyLM Workshop
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Jaap Jumelet, Tal Linzen, Aaron Mueller, Suchir Salhan, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox
Venues:
BabyLM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
65–74
Language:
URL:
https://aclanthology.org/2026.babylm-main.6/
DOI:
Bibkey:
Cite (ACL):
Pengfei Ren. 2026. Does Typological Distance Affect Cross-lingual Transfer?. In Proceedings of the Second BabyLM Workshop, pages 65–74, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Does Typological Distance Affect Cross-lingual Transfer? (Ren, BabyLM 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.babylm-main.6.pdf