@inproceedings{al-farib-etal-2026-state,
title = "State Tracking vs. Attention in a Morphologically Rich Low-Resource Language: A Controlled {B}angla Case Study with {M}amba-3",
author = "al Farib, Sahil and
Meem, Momota Ahsana and
Islam, Sheikh Redwanul and
Arefin, Khan Md. Shams and
Anik, Azizur Rahman",
editor = "Adelani, David Ifeoluwa and
Alabi, Jesujoba and
Arnett, Catherine and
Chang, Tyler and
Downey, C.M. and
Goldman, Omer and
Li, Senyu and
Miranda, Lester James Validad and
Qi, Jirui and
Stap, David and
Winata, Genta Indra and
Zhang, Xinyu Crystina",
booktitle = "Proceedings of the 6th Workshop on Multilingual Representation Learning ({MRL} 2026)",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.mrl-1.22/",
pages = "280--288",
ISBN = "979-8-89176-469-9",
abstract = "State space models such as Mamba-3 claim improved state tracking over attention at subquadratic cost, but this claim is validated almost entirely on English. We test whether it transfers to a morphologically rich, low resource setting by training parameter and token matched Transformer, Mamba-3, and hybrid language models from scratch on Bangla (24.5M nonembedding parameters, 1B tokens, five seeds per architecture). We evaluate on a new suite of 4{,}790 native speaker reviewed minimal pairs targeting Bangla subject verb person and honorific register agreement, and assess every comparison with Wilson confidence intervals and cross seed Welch $t$-tests. Across five seeds, the Mamba family achieves significantly lower perplexity than the Transformer (Welch p {\ensuremath{<}} 0.001). Transformer SVA accuracy declines from 91.3{\%} to 87.5{\%} as the intervener lengthens (paired $p=0.01$), whereas Mamba-3 shows no significant decline (83.3{\%} to 81.5{\%}, $p=0.22$); however, the direct difference between their distance slopes is not significant ($p=0.15$). A third finding is methodological rather than architectural: most single run between architecture agreement gaps, including a honorific register advantage significant under a within run McNemar test ($p = 0.0015$) do not survive replication once cross seed variance is included (Welch $p = 0.28$), because Mamba-3{'}s agreement accuracy is markedly more seed sensitive than the Transformer{'}s (cross seed standard deviation up to $\pm 8.4$ vs. $\pm 3.5$). At this scale, two seed comparisons of state space models can manufacture findings that vanish under replication. Perplexity does not predict distance behavior: the architectural signal we identify concerns robustness to distance and initialization, not a uniformly ``better'' architecture. We release code, probes, all fifteen checkpoints, and the analysis scripts."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="al-farib-etal-2026-state">
<titleInfo>
<title>State Tracking vs. Attention in a Morphologically Rich Low-Resource Language: A Controlled Bangla Case Study with Mamba-3</title>
</titleInfo>
<name type="personal">
<namePart type="given">Sahil</namePart>
<namePart type="family">al Farib</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Momota</namePart>
<namePart type="given">Ahsana</namePart>
<namePart type="family">Meem</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sheikh</namePart>
<namePart type="given">Redwanul</namePart>
<namePart type="family">Islam</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Khan</namePart>
<namePart type="given">Md.</namePart>
<namePart type="given">Shams</namePart>
<namePart type="family">Arefin</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Azizur</namePart>
<namePart type="given">Rahman</namePart>
<namePart type="family">Anik</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)</title>
</titleInfo>
<name type="personal">
<namePart type="given">David</namePart>
<namePart type="given">Ifeoluwa</namePart>
<namePart type="family">Adelani</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jesujoba</namePart>
<namePart type="family">Alabi</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Catherine</namePart>
<namePart type="family">Arnett</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Tyler</namePart>
<namePart type="family">Chang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">C</namePart>
<namePart type="given">M</namePart>
<namePart type="family">Downey</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Omer</namePart>
<namePart type="family">Goldman</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Senyu</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Lester</namePart>
<namePart type="given">James</namePart>
<namePart type="given">Validad</namePart>
<namePart type="family">Miranda</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jirui</namePart>
<namePart type="family">Qi</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">David</namePart>
<namePart type="family">Stap</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Genta</namePart>
<namePart type="given">Indra</namePart>
<namePart type="family">Winata</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xinyu</namePart>
<namePart type="given">Crystina</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-469-9</identifier>
</relatedItem>
<abstract>State space models such as Mamba-3 claim improved state tracking over attention at subquadratic cost, but this claim is validated almost entirely on English. We test whether it transfers to a morphologically rich, low resource setting by training parameter and token matched Transformer, Mamba-3, and hybrid language models from scratch on Bangla (24.5M nonembedding parameters, 1B tokens, five seeds per architecture). We evaluate on a new suite of 4,790 native speaker reviewed minimal pairs targeting Bangla subject verb person and honorific register agreement, and assess every comparison with Wilson confidence intervals and cross seed Welch t-tests. Across five seeds, the Mamba family achieves significantly lower perplexity than the Transformer (Welch p \ensuremath< 0.001). Transformer SVA accuracy declines from 91.3% to 87.5% as the intervener lengthens (paired p=0.01), whereas Mamba-3 shows no significant decline (83.3% to 81.5%, p=0.22); however, the direct difference between their distance slopes is not significant (p=0.15). A third finding is methodological rather than architectural: most single run between architecture agreement gaps, including a honorific register advantage significant under a within run McNemar test (p = 0.0015) do not survive replication once cross seed variance is included (Welch p = 0.28), because Mamba-3’s agreement accuracy is markedly more seed sensitive than the Transformer’s (cross seed standard deviation up to \pm 8.4 vs. \pm 3.5). At this scale, two seed comparisons of state space models can manufacture findings that vanish under replication. Perplexity does not predict distance behavior: the architectural signal we identify concerns robustness to distance and initialization, not a uniformly “better” architecture. We release code, probes, all fifteen checkpoints, and the analysis scripts.</abstract>
<identifier type="citekey">al-farib-etal-2026-state</identifier>
<location>
<url>https://aclanthology.org/2026.mrl-1.22/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>280</start>
<end>288</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T State Tracking vs. Attention in a Morphologically Rich Low-Resource Language: A Controlled Bangla Case Study with Mamba-3
%A al Farib, Sahil
%A Meem, Momota Ahsana
%A Islam, Sheikh Redwanul
%A Arefin, Khan Md. Shams
%A Anik, Azizur Rahman
%Y Adelani, David Ifeoluwa
%Y Alabi, Jesujoba
%Y Arnett, Catherine
%Y Chang, Tyler
%Y Downey, C. M.
%Y Goldman, Omer
%Y Li, Senyu
%Y Miranda, Lester James Validad
%Y Qi, Jirui
%Y Stap, David
%Y Winata, Genta Indra
%Y Zhang, Xinyu Crystina
%S Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-469-9
%F al-farib-etal-2026-state
%X State space models such as Mamba-3 claim improved state tracking over attention at subquadratic cost, but this claim is validated almost entirely on English. We test whether it transfers to a morphologically rich, low resource setting by training parameter and token matched Transformer, Mamba-3, and hybrid language models from scratch on Bangla (24.5M nonembedding parameters, 1B tokens, five seeds per architecture). We evaluate on a new suite of 4,790 native speaker reviewed minimal pairs targeting Bangla subject verb person and honorific register agreement, and assess every comparison with Wilson confidence intervals and cross seed Welch t-tests. Across five seeds, the Mamba family achieves significantly lower perplexity than the Transformer (Welch p \ensuremath< 0.001). Transformer SVA accuracy declines from 91.3% to 87.5% as the intervener lengthens (paired p=0.01), whereas Mamba-3 shows no significant decline (83.3% to 81.5%, p=0.22); however, the direct difference between their distance slopes is not significant (p=0.15). A third finding is methodological rather than architectural: most single run between architecture agreement gaps, including a honorific register advantage significant under a within run McNemar test (p = 0.0015) do not survive replication once cross seed variance is included (Welch p = 0.28), because Mamba-3’s agreement accuracy is markedly more seed sensitive than the Transformer’s (cross seed standard deviation up to \pm 8.4 vs. \pm 3.5). At this scale, two seed comparisons of state space models can manufacture findings that vanish under replication. Perplexity does not predict distance behavior: the architectural signal we identify concerns robustness to distance and initialization, not a uniformly “better” architecture. We release code, probes, all fifteen checkpoints, and the analysis scripts.
%U https://aclanthology.org/2026.mrl-1.22/
%P 280-288
Markdown (Informal)
[State Tracking vs. Attention in a Morphologically Rich Low-Resource Language: A Controlled Bangla Case Study with Mamba-3](https://aclanthology.org/2026.mrl-1.22/) (al Farib et al., MRL 2026)
ACL