Hanyu Liao
Author directory2026
When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM
Hanyu Liao | Yijing Chen | Tao Ji
Proceedings of the Second BabyLM Workshop
Hanyu Liao | Yijing Chen | Tao Ji
Proceedings of the Second BabyLM Workshop
Multilingual BabyLM systems typically fix language proportions statically while overlooking that the weakest language changes throughout training. We present ARMOR-DP, a modular diagnostic framework that identifies linguistic weakness arising from representation, exposure, capacity, and objective pressure, and dynamically adjusts sampling ratios according to per language validation loss. Controlled experiments show that equal character budgets lead to severe collapse in Chinese. Byte level representation resolves this issue, whereas increased exposure to Chinese subsequently shifts the bottleneck to English. ARMOR-DP tracks these transitions throughout training and maintains a better balance than fixed Chinese protection. On the official evaluation suite, our 100M parameter curriculum dynamic d768 checkpoint achieves 0.503 on the zero shot metric and 0.387 on the combined metric, attaining a multilingual leaderboard average of 38.32 with per language scores of 35.34 for English, 40.96 for Dutch, and 38.65 for Chinese. Dutch and Chinese substantially outperform the baseline, while English exhibits only a modest decrease, indicating that the overall model capacity is well preserved. On the POS task, the model records scores of 85.95% for English, 88.13% for Dutch, and 82.29% for Chinese, compared with official baselines of 84.7% on English MultiBLiMP, 92.4% on Dutch MultiBLiMP, and 78.5% on ZhoBLiMP, demonstrating strong morphological and syntactic competence. A matched d512 variant obtains 0.381 on the combined metric, outperforming static ZH50 at 0.359 while requiring only 6.46 Chinese corpus epochs instead of 9.78. Our findings suggest that multilingual BabyLM systems should treat linguistic weakness as a training time state rather than a fixed deficit.