<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="airlangga-etal-2026-sparse">
    <titleInfo>
        <title>Sparse Long Memory and Transition Bottlenecks Shape Positional Recall in Mamba Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Muhammad</namePart>
        <namePart type="given">Cendekia</namePart>
        <namePart type="family">Airlangga</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hilal</namePart>
        <namePart type="family">AlQuabeh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Munachiso</namePart>
        <namePart type="given">Samuel</namePart>
        <namePart type="family">Nwadike</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kentaro</namePart>
        <namePart type="family">Inui</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>Mamba language models process sequences recurrently, but recurrent processing does not imply that all tokens are retained equally. We study this gap using structured recall tasks that expose primacy, recency, and lost-in-the-middle behavior. Our results suggest that Mamba uses two functional memory regimes: a short-lived state that supports recency, and a sparse long-lived channel subspace that preserves selected early inputs. Targeted interventions on this long-memory subspace selectively degrade early recall and transfer to long-context QA and summarization, indicating that the same channels support real long-context use. We then ask why middle items are forgotten despite being processed sequentially. We find that the central failure is a short-to-long memory transition bottleneck: middle items are no longer recent enough for short memory, but are often not promoted into the sparse long-lived subspace. Repeated input structure strengthens this bottleneck by changing the input-dependent discretization dynamics. These findings provide a mechanistic account of Mamba’s U-shaped recall curve as the interaction between long memory, short memory, and a fragile transition between them.</abstract>
    <identifier type="citekey">airlangga-etal-2026-sparse</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.1944/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>41448</start>
            <end>41463</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Sparse Long Memory and Transition Bottlenecks Shape Positional Recall in Mamba Language Models
%A Airlangga, Muhammad Cendekia
%A AlQuabeh, Hilal
%A Nwadike, Munachiso Samuel
%A Inui, Kentaro
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F airlangga-etal-2026-sparse
%X Mamba language models process sequences recurrently, but recurrent processing does not imply that all tokens are retained equally. We study this gap using structured recall tasks that expose primacy, recency, and lost-in-the-middle behavior. Our results suggest that Mamba uses two functional memory regimes: a short-lived state that supports recency, and a sparse long-lived channel subspace that preserves selected early inputs. Targeted interventions on this long-memory subspace selectively degrade early recall and transfer to long-context QA and summarization, indicating that the same channels support real long-context use. We then ask why middle items are forgotten despite being processed sequentially. We find that the central failure is a short-to-long memory transition bottleneck: middle items are no longer recent enough for short memory, but are often not promoted into the sparse long-lived subspace. Repeated input structure strengthens this bottleneck by changing the input-dependent discretization dynamics. These findings provide a mechanistic account of Mamba’s U-shaped recall curve as the interaction between long memory, short memory, and a fragile transition between them.
%U https://aclanthology.org/2026.findings-emnlp.1944/
%P 41448-41463

Markdown (Informal)

[Sparse Long Memory and Transition Bottlenecks Shape Positional Recall in Mamba Language Models](https://aclanthology.org/2026.findings-emnlp.1944/) (Airlangga et al., Findings 2026)

• Sparse Long Memory and Transition Bottlenecks Shape Positional Recall in Mamba Language Models (Airlangga et al., Findings 2026)

ACL

• Muhammad Cendekia Airlangga, Hilal AlQuabeh, Munachiso Samuel Nwadike, and Kentaro Inui. 2026. Sparse Long Memory and Transition Bottlenecks Shape Positional Recall in Mamba Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 41448–41463, Budapest, Hungary. Association for Computational Linguistics.