<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="liao-etal-2026-weakest">
    <titleInfo>
        <title>When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hanyu</namePart>
        <namePart type="family">Liao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yijing</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tao</namePart>
        <namePart type="family">Ji</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second BabyLM Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Leshem</namePart>
            <namePart type="family">Choshen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ryan</namePart>
            <namePart type="family">Cotterell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mustafa</namePart>
            <namePart type="given">Omer</namePart>
            <namePart type="family">Gul</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jaap</namePart>
            <namePart type="family">Jumelet</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tal</namePart>
            <namePart type="family">Linzen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aaron</namePart>
            <namePart type="family">Mueller</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Suchir</namePart>
            <namePart type="family">Salhan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Raj</namePart>
            <namePart type="given">Sanjay</namePart>
            <namePart type="family">Shah</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alex</namePart>
            <namePart type="family">Warstadt</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ethan</namePart>
            <namePart type="given">Gotlieb</namePart>
            <namePart type="family">Wilcox</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-475-0</identifier>
    </relatedItem>
    <abstract>Multilingual BabyLM systems typically fix language proportions statically while overlooking that the weakest language changes throughout training. We present ARMOR-DP, a modular diagnostic framework that identifies linguistic weakness arising from representation, exposure, capacity, and objective pressure, and dynamically adjusts sampling ratios according to per language validation loss. Controlled experiments show that equal character budgets lead to severe collapse in Chinese. Byte level representation resolves this issue, whereas increased exposure to Chinese subsequently shifts the bottleneck to English. ARMOR-DP tracks these transitions throughout training and maintains a better balance than fixed Chinese protection. On the official evaluation suite, our 100M parameter curriculum dynamic d768 checkpoint achieves 0.503 on the zero shot metric and 0.387 on the combined metric, attaining a multilingual leaderboard average of 38.32 with per language scores of 35.34 for English, 40.96 for Dutch, and 38.65 for Chinese. Dutch and Chinese substantially outperform the baseline, while English exhibits only a modest decrease, indicating that the overall model capacity is well preserved. On the POS task, the model records scores of 85.95% for English, 88.13% for Dutch, and 82.29% for Chinese, compared with official baselines of 84.7% on English MultiBLiMP, 92.4% on Dutch MultiBLiMP, and 78.5% on ZhoBLiMP, demonstrating strong morphological and syntactic competence. A matched d512 variant obtains 0.381 on the combined metric, outperforming static ZH50 at 0.359 while requiring only 6.46 Chinese corpus epochs instead of 9.78. Our findings suggest that multilingual BabyLM systems should treat linguistic weakness as a training time state rather than a fixed deficit.</abstract>
    <identifier type="citekey">liao-etal-2026-weakest</identifier>
    <location>
        <url>https://aclanthology.org/2026.babylm-main.25/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>309</start>
            <end>319</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM
%A Liao, Hanyu
%A Chen, Yijing
%A Ji, Tao
%Y Choshen, Leshem
%Y Cotterell, Ryan
%Y Gul, Mustafa Omer
%Y Jumelet, Jaap
%Y Linzen, Tal
%Y Mueller, Aaron
%Y Salhan, Suchir
%Y Shah, Raj Sanjay
%Y Warstadt, Alex
%Y Wilcox, Ethan Gotlieb
%S Proceedings of the Second BabyLM Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-475-0
%F liao-etal-2026-weakest
%X Multilingual BabyLM systems typically fix language proportions statically while overlooking that the weakest language changes throughout training. We present ARMOR-DP, a modular diagnostic framework that identifies linguistic weakness arising from representation, exposure, capacity, and objective pressure, and dynamically adjusts sampling ratios according to per language validation loss. Controlled experiments show that equal character budgets lead to severe collapse in Chinese. Byte level representation resolves this issue, whereas increased exposure to Chinese subsequently shifts the bottleneck to English. ARMOR-DP tracks these transitions throughout training and maintains a better balance than fixed Chinese protection. On the official evaluation suite, our 100M parameter curriculum dynamic d768 checkpoint achieves 0.503 on the zero shot metric and 0.387 on the combined metric, attaining a multilingual leaderboard average of 38.32 with per language scores of 35.34 for English, 40.96 for Dutch, and 38.65 for Chinese. Dutch and Chinese substantially outperform the baseline, while English exhibits only a modest decrease, indicating that the overall model capacity is well preserved. On the POS task, the model records scores of 85.95% for English, 88.13% for Dutch, and 82.29% for Chinese, compared with official baselines of 84.7% on English MultiBLiMP, 92.4% on Dutch MultiBLiMP, and 78.5% on ZhoBLiMP, demonstrating strong morphological and syntactic competence. A matched d512 variant obtains 0.381 on the combined metric, outperforming static ZH50 at 0.359 while requiring only 6.46 Chinese corpus epochs instead of 9.78. Our findings suggest that multilingual BabyLM systems should treat linguistic weakness as a training time state rather than a fixed deficit.
%U https://aclanthology.org/2026.babylm-main.25/
%P 309-319

Markdown (Informal)

[When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM](https://aclanthology.org/2026.babylm-main.25/) (Liao et al., BabyLM 2026)

• When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM (Liao et al., BabyLM 2026)

ACL

• Hanyu Liao, Yijing Chen, and Tao Ji. 2026. When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM. In Proceedings of the Second BabyLM Workshop, pages 309–319, Budapest, Hungary. Association for Computational Linguistics.