<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="matsuda-asahara-2026-probing">
    <titleInfo>
        <title>Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hiroshi</namePart>
        <namePart type="family">Matsuda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Masayuki</namePart>
        <namePart type="family">Asahara</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Ninth Workshop on Universal Dependencies (UDW 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Çağrı</namePart>
            <namePart type="family">Çöltekin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kaja</namePart>
            <namePart type="family">Dobrovoljc</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma de Mallorca, Spain</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this research, we introduce LoRA probing, a lightweight approach for observing how core syntactic abilities emerge during LLM pretraining. Leveraging OLMo-2’s public intermediate checkpoints, we trace learning curves across 24 pretraining stages on 33 Universal Dependencies languages by fine-tuning LoRA with step-by-step parsing instructions and a simple tabular output. To fit the relatively short context length of the OLMo-2, we design a compact 2-step-no-form prompt template and this matches the baseline in average accuracy while halving the context length and substantially increasing throughput, enabling efficient large-scale evaluation. Token Recall surpasses 0.9 within the first 1–2K pretraining steps, indicating that stable output formatting emerges early. Despite OLMo-2-7B’s English-centric pretraining, LAS exceeds 80 points in 29 of 33 languages; however, relations such as iobj and csubj show delayed onset and instability across many languages. LoRA probing thus provides a practical, reproducible lens on the cross-lingual dynamics of syntactic acquisition during LLM pretraining.</abstract>
    <identifier type="citekey">matsuda-asahara-2026-probing</identifier>
    <identifier type="doi">10.63317/2n8gkp49p3wg</identifier>
    <location>
        <url>https://aclanthology.org/2026.udw-1.1/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>1</start>
            <end>48</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining
%A Matsuda, Hiroshi
%A Asahara, Masayuki
%Y Çöltekin, Çağrı
%Y Dobrovoljc, Kaja
%S Proceedings of the Ninth Workshop on Universal Dependencies (UDW 2026)
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma de Mallorca, Spain
%F matsuda-asahara-2026-probing
%X In this research, we introduce LoRA probing, a lightweight approach for observing how core syntactic abilities emerge during LLM pretraining. Leveraging OLMo-2’s public intermediate checkpoints, we trace learning curves across 24 pretraining stages on 33 Universal Dependencies languages by fine-tuning LoRA with step-by-step parsing instructions and a simple tabular output. To fit the relatively short context length of the OLMo-2, we design a compact 2-step-no-form prompt template and this matches the baseline in average accuracy while halving the context length and substantially increasing throughput, enabling efficient large-scale evaluation. Token Recall surpasses 0.9 within the first 1–2K pretraining steps, indicating that stable output formatting emerges early. Despite OLMo-2-7B’s English-centric pretraining, LAS exceeds 80 points in 29 of 33 languages; however, relations such as iobj and csubj show delayed onset and instability across many languages. LoRA probing thus provides a practical, reproducible lens on the cross-lingual dynamics of syntactic acquisition during LLM pretraining.
%R 10.63317/2n8gkp49p3wg
%U https://aclanthology.org/2026.udw-1.1/
%U https://doi.org/10.63317/2n8gkp49p3wg
%P 1-48

Markdown (Informal)

[Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining](https://aclanthology.org/2026.udw-1.1/) (Matsuda & Asahara, UDW 2026)

• Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining (Matsuda & Asahara, UDW 2026)

ACL

• Hiroshi Matsuda and Masayuki Asahara. 2026. Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining. In Proceedings of the Ninth Workshop on Universal Dependencies (UDW 2026), pages 1–48, Palma de Mallorca, Spain. ELRA Language Resources Association (ELRA).