<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kadlcik-stefanik-2024-self">
    <titleInfo>
        <title>Self-training Language Models for Arithmetic Reasoning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Marek</namePart>
        <namePart type="family">Kadlčík</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Michal</namePart>
        <namePart type="family">Štefánik</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent language models achieve impressive results in tasks involving complex multistep reasoning, but scaling these capabilities further traditionally requires expensive collection of more annotated data.In this work, we explore the potential of improving models’ reasoning capabilities without new data, merely using automated feedback to the validity of their predictions in arithmetic reasoning (self-training).In systematic experimentation across six different arithmetic reasoning datasets, we find that models can substantially improve in both single-round (offline) and online self-training, reaching a correct result in +13.9% and +25.9% more cases, respectively, underlining the importance of actuality of self-training feedback. We further find that in the single-round, offline self-training, traditional supervised training can deliver gains comparable to preference optimization, but in online self-training, preference optimization methods largely outperform supervised training thanks to their superior stability and robustness on unseen types of problems.</abstract>
    <identifier type="citekey">kadlcik-stefanik-2024-self</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-emnlp.721</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>12378</start>
            <end>12386</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Self-training Language Models for Arithmetic Reasoning
%A Kadlčík, Marek
%A Štefánik, Michal
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Findings of the Association for Computational Linguistics: EMNLP 2024
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F kadlcik-stefanik-2024-self
%X Recent language models achieve impressive results in tasks involving complex multistep reasoning, but scaling these capabilities further traditionally requires expensive collection of more annotated data.In this work, we explore the potential of improving models’ reasoning capabilities without new data, merely using automated feedback to the validity of their predictions in arithmetic reasoning (self-training).In systematic experimentation across six different arithmetic reasoning datasets, we find that models can substantially improve in both single-round (offline) and online self-training, reaching a correct result in +13.9% and +25.9% more cases, respectively, underlining the importance of actuality of self-training feedback. We further find that in the single-round, offline self-training, traditional supervised training can deliver gains comparable to preference optimization, but in online self-training, preference optimization methods largely outperform supervised training thanks to their superior stability and robustness on unseen types of problems.
%U https://aclanthology.org/2024.findings-emnlp.721
%P 12378-12386

Markdown (Informal)

[Self-training Language Models for Arithmetic Reasoning](https://aclanthology.org/2024.findings-emnlp.721) (Kadlčík & Štefánik, Findings 2024)

• Self-training Language Models for Arithmetic Reasoning (Kadlčík & Štefánik, Findings 2024)

ACL

• Marek Kadlčík and Michal Štefánik. 2024. Self-training Language Models for Arithmetic Reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12378–12386, Miami, Florida, USA. Association for Computational Linguistics.