<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="patil-etal-2026-beyond">
    <titleInfo>
        <title>Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Parth</namePart>
        <namePart type="given">Shrikant</namePart>
        <namePart type="family">Patil</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yash</namePart>
        <namePart type="family">Sinha</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Murari</namePart>
        <namePart type="family">Mandal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dhruv</namePart>
        <namePart type="family">Kumar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marco</namePart>
            <namePart type="family">Valentino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Deborah</namePart>
            <namePart type="family">Ferreira</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mokanarangan</namePart>
            <namePart type="family">Thayaparan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Leonardo</namePart>
            <namePart type="family">Ranaldi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jo-Ku</namePart>
            <namePart type="family">Cheng</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Freitas</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-468-2</identifier>
    </relatedItem>
    <abstract>Aggregate mathematical accuracy does not usually identify the source of a model failure. In algebraic reasoning, errors may arise from nesting depth, operator identity, simultaneous intermediate values, branch-local composition, repeated-token counting, sequential dependence, or numeric magnitude. We introduce a nine-dimension algebraic complexity framework that converts these factors into controlled diagnostic variables. For each dimension, a parametric generator increases one target variable while holding the remaining variables at minimum values. Every retained problem is verified by a computer algebra system. We evaluate eight models, including six standard instruction- tuned models and two reasoning-specialised models. The results show that standard models exhibit a scale-limited D4 working-memory collapse in the 20–40 parallel-branch region. Notably, Qwen3 235B does not escape the transition band observed for Llama 3 8B, despite its much larger parameter scale. Reasoning- specialised models shift this limit to higher levels, but they do not remove it. D5 branch-local composition and D7 repeated-token counting produce distinct failures that are not reducible to aggregate length or depth.</abstract>
    <identifier type="citekey">patil-etal-2026-beyond</identifier>
    <location>
        <url>https://aclanthology.org/2026.mathnlp-main.17/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>216</start>
            <end>239</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions
%A Patil, Parth Shrikant
%A Sinha, Yash
%A Mandal, Murari
%A Kumar, Dhruv
%Y Valentino, Marco
%Y Ferreira, Deborah
%Y Thayaparan, Mokanarangan
%Y Ranaldi, Leonardo
%Y Cheng, Jo-Ku
%Y Freitas, Andre
%S The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-468-2
%F patil-etal-2026-beyond
%X Aggregate mathematical accuracy does not usually identify the source of a model failure. In algebraic reasoning, errors may arise from nesting depth, operator identity, simultaneous intermediate values, branch-local composition, repeated-token counting, sequential dependence, or numeric magnitude. We introduce a nine-dimension algebraic complexity framework that converts these factors into controlled diagnostic variables. For each dimension, a parametric generator increases one target variable while holding the remaining variables at minimum values. Every retained problem is verified by a computer algebra system. We evaluate eight models, including six standard instruction- tuned models and two reasoning-specialised models. The results show that standard models exhibit a scale-limited D4 working-memory collapse in the 20–40 parallel-branch region. Notably, Qwen3 235B does not escape the transition band observed for Llama 3 8B, despite its much larger parameter scale. Reasoning- specialised models shift this limit to higher levels, but they do not remove it. D5 branch-local composition and D7 repeated-token counting produce distinct failures that are not reducible to aggregate length or depth.
%U https://aclanthology.org/2026.mathnlp-main.17/
%P 216-239

Markdown (Informal)

[Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions](https://aclanthology.org/2026.mathnlp-main.17/) (Patil et al., MathNLP 2026)

• Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions (Patil et al., MathNLP 2026)

ACL

• Parth Shrikant Patil, Yash Sinha, Murari Mandal, and Dhruv Kumar. 2026. Beyond Accuracy: Diagnosing Algebraic Reasoning Failures in LLMs Across Nine Complexity Dimensions. In The Proceedings of the 4th Workshop on Mathematical Natural Language Processing (MathNLP 2026), pages 216–239, Budapest, Hungary. Association for Computational Linguistics.