<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="thamma-heilbron-2026-human">
    <titleInfo>
        <title>Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Abishek</namePart>
        <namePart type="family">Thamma</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Micha</namePart>
        <namePart type="family">Heilbron</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Human memory is fleeting. As words are processed, the exact wordforms that make up incoming sentences are rapidly lost. Cognitive scientists have long believed that this limitation of working memory may, paradoxically, help in learning language – an idea supported by classic connectionist modelling work. The rise of Transformers appears to challenge this idea, as these models can learn language effectively, despite lacking working memory limitations or other architectural recency biases. Here, we investigate the hypothesized benefit of fleeting memory for language learning in tightly controlled experiments on transformer language models. Training transformers with and without fleeting memory on a developmentally realistic training set, we find that fleeting memory consistently improves language learning (as quantified by both overall language modelling performance and targeted syntactic evaluation) but, unexpectedly, impairs surprisal-based prediction of human reading times. Interestingly, follow up analyses revealed that this discrepancy – better language modeling, yet worse reading time prediction – could not be accounted for by prior explanations of why better language models sometimes fit human reading time worse. Together, these results support a benefit of memory limitations on neural network language learning – but not on predicting behavior.</abstract>
    <identifier type="citekey">thamma-heilbron-2026-human</identifier>
    <identifier type="doi">10.1162/tacl.a.688</identifier>
    <location>
        <url>https://aclanthology.org/2026.tacl-1.39/</url>
    </location>
    <part>
        <date>2026</date>
        <detail type="volume"><number>14</number></detail>
        <extent unit="page">
            <start>877</start>
            <end>892</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models
%A Thamma, Abishek
%A Heilbron, Micha
%J Transactions of the Association for Computational Linguistics
%D 2026
%V 14
%I MIT Press
%C Cambridge, MA
%F thamma-heilbron-2026-human
%X Human memory is fleeting. As words are processed, the exact wordforms that make up incoming sentences are rapidly lost. Cognitive scientists have long believed that this limitation of working memory may, paradoxically, help in learning language – an idea supported by classic connectionist modelling work. The rise of Transformers appears to challenge this idea, as these models can learn language effectively, despite lacking working memory limitations or other architectural recency biases. Here, we investigate the hypothesized benefit of fleeting memory for language learning in tightly controlled experiments on transformer language models. Training transformers with and without fleeting memory on a developmentally realistic training set, we find that fleeting memory consistently improves language learning (as quantified by both overall language modelling performance and targeted syntactic evaluation) but, unexpectedly, impairs surprisal-based prediction of human reading times. Interestingly, follow up analyses revealed that this discrepancy – better language modeling, yet worse reading time prediction – could not be accounted for by prior explanations of why better language models sometimes fit human reading time worse. Together, these results support a benefit of memory limitations on neural network language learning – but not on predicting behavior.
%R 10.1162/tacl.a.688
%U https://aclanthology.org/2026.tacl-1.39/
%U https://doi.org/10.1162/tacl.a.688
%P 877-892

Markdown (Informal)

[Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models](https://aclanthology.org/2026.tacl-1.39/) (Thamma & Heilbron, TACL 2026)

• Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models (Thamma & Heilbron, TACL 2026)

ACL

• Abishek Thamma and Micha Heilbron. 2026. Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models. Transactions of the Association for Computational Linguistics, 14:877–892.