<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2026-mua">
    <titleInfo>
        <title>MUA: Multiple Context Units Attention For Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zhiyi</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yaming</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kai</namePart>
        <namePart type="family">Shuang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yiren</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Large language models (LLMs) are increasingly required to process inputs composed of multiple context units, such as answer options, retrieved passages, or in-context demonstrations. Effective reasoning in such scenarios demands capturing both intrinsic semantics within each unit and the aggregated semantics across units. However, recent efforts typically treat multiple units as a single entangled sequence, ignoring the distinct knowledge within isolated units, which leads to suboptimal performance. To this end, we propose MUA(Multiple Context Units Attention), a dual-view self-attention module that computes a local view for learning unit-specific semantics and a global view to enable bidirectional interaction across units. These two views share projection matrices and are fused by a lightweight input-conditioned gate, while generation proceeds with standard causal decoding. With an optimized implementation, this design achieves efficient fusion of multiple context units and superior information utilization during input encoding at marginal overhead. Experiments on diverse multiple context units reasoning benchmarks, including multiple-choice, multi-hop QA and language understanding tasks, show that MUA consistently improves performance across model scales, exhibits increased robustness to context units reordering, and establishes new state-of-the-art results on key benchmarks.</abstract>
    <identifier type="citekey">li-etal-2026-mua</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1969/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>44822</start>
            <end>44833</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T MUA: Multiple Context Units Attention For Large Language Models
%A Li, Zhiyi
%A Yang, Yaming
%A Shuang, Kai
%A Chen, Yiren
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F li-etal-2026-mua
%X Large language models (LLMs) are increasingly required to process inputs composed of multiple context units, such as answer options, retrieved passages, or in-context demonstrations. Effective reasoning in such scenarios demands capturing both intrinsic semantics within each unit and the aggregated semantics across units. However, recent efforts typically treat multiple units as a single entangled sequence, ignoring the distinct knowledge within isolated units, which leads to suboptimal performance. To this end, we propose MUA(Multiple Context Units Attention), a dual-view self-attention module that computes a local view for learning unit-specific semantics and a global view to enable bidirectional interaction across units. These two views share projection matrices and are fused by a lightweight input-conditioned gate, while generation proceeds with standard causal decoding. With an optimized implementation, this design achieves efficient fusion of multiple context units and superior information utilization during input encoding at marginal overhead. Experiments on diverse multiple context units reasoning benchmarks, including multiple-choice, multi-hop QA and language understanding tasks, show that MUA consistently improves performance across model scales, exhibits increased robustness to context units reordering, and establishes new state-of-the-art results on key benchmarks.
%U https://aclanthology.org/2026.emnlp-main.1969/
%P 44822-44833

Markdown (Informal)

[MUA: Multiple Context Units Attention For Large Language Models](https://aclanthology.org/2026.emnlp-main.1969/) (Li et al., EMNLP 2026)

• MUA: Multiple Context Units Attention For Large Language Models (Li et al., EMNLP 2026)

ACL

• Zhiyi Li, Yaming Yang, Kai Shuang, and Yiren Chen. 2026. MUA: Multiple Context Units Attention For Large Language Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 44822–44833, Budapest, Hungary. Association for Computational Linguistics.