JiWoo Hwang

Author directory

2026

We investigate the effects of multilingual pre-training with limited data by addressing the problem of how language exposure should be distributed and ordered across English, Dutch, and Chinese. In particular, we compare balanced multilingual exposure to language-dominant exposure, where one language is exposed with more training data than the others. We also compare mixed training, where multilingual training data is used throughout training, with curriculum training, where monolingual exposure of the dominant language is followed by balanced multilingual exposure. Our results show that multilingual BabyLM performance varies with both the amount and order of language exposure. Mixed training outperforms curriculum learning across all 15 matched configurations, indicating that simultaneous multilingual exposure is more effective than staged exposure in our setting. For each primary language, the best language-dominant mixed training configuration outperforms the balanced configuration, suggesting that an equal distribution is not necessarily optimal under a limited data budget. Interestingly, the Dutch-primary configurations achieve the best overall performance.