<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lyu-etal-2026-structural">
    <titleInfo>
        <title>Structural Fabric of Data Shapes the Geometry of LLM Training</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xinyu</namePart>
        <namePart type="family">Lyu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hanqi</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shenghan</namePart>
        <namePart type="family">Zuo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kai</namePart>
        <namePart type="family">Yu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lu</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>The performance of large language models is tied to the quality and organization of their training data. However, existing data metrics emphasize surface-level properties, such as sequence length or model-based quality scores, and provide limited insight into how the internal organization of data relates to optimization. We study the structural fabric of training data, defined as the pattern of semantic dependencies within a sequence. We quantify this property using dependency density, a normalized measure of effective segment-level dependencies. Across data length, domain, and format, we find that dependency density varies with data configuration: code shows higher dependency density than natural language within long-context data, and instruction-tuning mappings show higher dependency density than raw intra-input modeling. We show that coherent structural fabric is associated with a lower gradient effective rank, indicating a more concentrated optimization geometry with reduced gradient interference. We observe that this low-rank optimization regime is associated with a more stable intrinsic dimension, preserving a compact representation manifold that may mitigate representation noise and support downstream performance. These results provide a geometric perspective on how data structure relates to LLM training, linking data configuration, optimization dynamics, and model behavior through the lens of structural fabric.</abstract>
    <identifier type="citekey">lyu-etal-2026-structural</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.298/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>6312</start>
            <end>6324</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Structural Fabric of Data Shapes the Geometry of LLM Training
%A Lyu, Xinyu
%A Li, Hanqi
%A Zuo, Shenghan
%A Yu, Kai
%A Chen, Lu
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F lyu-etal-2026-structural
%X The performance of large language models is tied to the quality and organization of their training data. However, existing data metrics emphasize surface-level properties, such as sequence length or model-based quality scores, and provide limited insight into how the internal organization of data relates to optimization. We study the structural fabric of training data, defined as the pattern of semantic dependencies within a sequence. We quantify this property using dependency density, a normalized measure of effective segment-level dependencies. Across data length, domain, and format, we find that dependency density varies with data configuration: code shows higher dependency density than natural language within long-context data, and instruction-tuning mappings show higher dependency density than raw intra-input modeling. We show that coherent structural fabric is associated with a lower gradient effective rank, indicating a more concentrated optimization geometry with reduced gradient interference. We observe that this low-rank optimization regime is associated with a more stable intrinsic dimension, preserving a compact representation manifold that may mitigate representation noise and support downstream performance. These results provide a geometric perspective on how data structure relates to LLM training, linking data configuration, optimization dynamics, and model behavior through the lens of structural fabric.
%U https://aclanthology.org/2026.findings-emnlp.298/
%P 6312-6324

Markdown (Informal)

[Structural Fabric of Data Shapes the Geometry of LLM Training](https://aclanthology.org/2026.findings-emnlp.298/) (Lyu et al., Findings 2026)

• Structural Fabric of Data Shapes the Geometry of LLM Training (Lyu et al., Findings 2026)

ACL

• Xinyu Lyu, Hanqi Li, Shenghan Zuo, Kai Yu, and Lu Chen. 2026. Structural Fabric of Data Shapes the Geometry of LLM Training. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 6312–6324, Budapest, Hungary. Association for Computational Linguistics.