<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="prudnikov-kachan-2026-cross">
    <titleInfo>
        <title>Cross-Attention Eigenvalues Know the Word Order in Multilingual Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nikolai</namePart>
        <namePart type="family">Prudnikov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Oleg</namePart>
        <namePart type="family">Kachan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Multilingual machine translation models must adapt to the grammatical structure of the target language, yet how structural information shapes decoder computation remains unclear. Using the eigenvalue spectra of per-(layer, head) attention Gram matrices instead of raw attention weights, we show that cross-attention geometry in NLLB-200 is organized around the target language: these spectral features carry a strong, distributed signal about target-language word order (SVO vs. SOV) when English is the source, with near-perfect decodability across cross-validation. The signal is not concentrated in a small subset of heads, but is broadly distributed across layers and heads—yet it degrades sharply at every level of aggregation when English is the target. This asymmetry suggests that cross-attention geometry reflects the demands of target-side generation rather than source-side structure. Leave-one-language-out evaluation confirms that the signal generalizes to unseen languages, ruling out language identity as a confound.</abstract>
    <identifier type="citekey">prudnikov-kachan-2026-cross</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.31/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>650</start>
            <end>656</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Cross-Attention Eigenvalues Know the Word Order in Multilingual Machine Translation
%A Prudnikov, Nikolai
%A Kachan, Oleg
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F prudnikov-kachan-2026-cross
%X Multilingual machine translation models must adapt to the grammatical structure of the target language, yet how structural information shapes decoder computation remains unclear. Using the eigenvalue spectra of per-(layer, head) attention Gram matrices instead of raw attention weights, we show that cross-attention geometry in NLLB-200 is organized around the target language: these spectral features carry a strong, distributed signal about target-language word order (SVO vs. SOV) when English is the source, with near-perfect decodability across cross-validation. The signal is not concentrated in a small subset of heads, but is broadly distributed across layers and heads—yet it degrades sharply at every level of aggregation when English is the target. This asymmetry suggests that cross-attention geometry reflects the demands of target-side generation rather than source-side structure. Leave-one-language-out evaluation confirms that the signal generalizes to unseen languages, ruling out language identity as a confound.
%U https://aclanthology.org/2026.emnlp-main.31/
%P 650-656

Markdown (Informal)

[Cross-Attention Eigenvalues Know the Word Order in Multilingual Machine Translation](https://aclanthology.org/2026.emnlp-main.31/) (Prudnikov & Kachan, EMNLP 2026)

• Cross-Attention Eigenvalues Know the Word Order in Multilingual Machine Translation (Prudnikov & Kachan, EMNLP 2026)

ACL

• Nikolai Prudnikov and Oleg Kachan. 2026. Cross-Attention Eigenvalues Know the Word Order in Multilingual Machine Translation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 650–656, Budapest, Hungary. Association for Computational Linguistics.