<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="han-etal-2026-revisiting">
    <titleInfo>
        <title>Revisiting Task Vectors: A Structural Analysis of Their Limits</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Leekyeung</namePart>
        <namePart type="family">Han</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hyeseung</namePart>
        <namePart type="family">JI</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sangwon</namePart>
        <namePart type="family">Jung</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Seungwoo</namePart>
        <namePart type="family">Lyu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paul</namePart>
        <namePart type="given">Hongsuck</namePart>
        <namePart type="family">Seo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>To mitigate high inference costs for in-context learning, prior works have proposed activation-space task vectors that encapsulate demonstration contexts into static representations. However, performance occasionally degrades across tasks, leaving it unclear whether this stems from sub-optimal extraction heuristics or fundamental limitations. We resolve this by decomposing Transformer self-attention to establish a theoretical best-case approximation: the Minimal-Gap Approximation. By isolating structural constraints from heuristic flaws, the Minimal-Gap Approximation reveals that task vectors are fundamentally bottlenecked by attention and length gaps. Specifically, because task vectors lock contextual information to fixed attention maps and positional alignments, they structurally fail under dynamic routing, length mismatches, and multi-token generation. We further show that these findings consistently hold across training-free extract-and-inject task vector methods, model architectures, and model scales.</abstract>
    <identifier type="citekey">han-etal-2026-revisiting</identifier>
    <identifier type="doi">10.18653/2026.findings-emnlp.1606</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.1606/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>34196</start>
            <end>34212</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Revisiting Task Vectors: A Structural Analysis of Their Limits
%A Han, Leekyeung
%A JI, Hyeseung
%A Jung, Sangwon
%A Lyu, Seungwoo
%A Seo, Paul Hongsuck
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F han-etal-2026-revisiting
%X To mitigate high inference costs for in-context learning, prior works have proposed activation-space task vectors that encapsulate demonstration contexts into static representations. However, performance occasionally degrades across tasks, leaving it unclear whether this stems from sub-optimal extraction heuristics or fundamental limitations. We resolve this by decomposing Transformer self-attention to establish a theoretical best-case approximation: the Minimal-Gap Approximation. By isolating structural constraints from heuristic flaws, the Minimal-Gap Approximation reveals that task vectors are fundamentally bottlenecked by attention and length gaps. Specifically, because task vectors lock contextual information to fixed attention maps and positional alignments, they structurally fail under dynamic routing, length mismatches, and multi-token generation. We further show that these findings consistently hold across training-free extract-and-inject task vector methods, model architectures, and model scales.
%R 10.18653/2026.findings-emnlp.1606
%U https://aclanthology.org/2026.findings-emnlp.1606/
%U https://doi.org/10.18653/2026.findings-emnlp.1606
%P 34196-34212

Markdown (Informal)

[Revisiting Task Vectors: A Structural Analysis of Their Limits](https://aclanthology.org/2026.findings-emnlp.1606/) (Han et al., Findings 2026)

• Revisiting Task Vectors: A Structural Analysis of Their Limits (Han et al., Findings 2026)

ACL

• Leekyeung Han, Hyeseung JI, Sangwon Jung, Seungwoo Lyu, and Paul Hongsuck Seo. 2026. Revisiting Task Vectors: A Structural Analysis of Their Limits. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 34196–34212, Budapest, Hungary. Association for Computational Linguistics.