<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pal-baral-2021-investigating-numeracy">
    <titleInfo>
        <title>Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kuntal</namePart>
        <namePart type="given">Kumar</namePart>
        <namePart type="family">Pal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chitta</namePart>
        <namePart type="family">Baral</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2021</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marie-Francine</namePart>
            <namePart type="family">Moens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Xuanjing</namePart>
            <namePart type="family">Huang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lucia</namePart>
            <namePart type="family">Specia</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Scott</namePart>
            <namePart type="given">Wen-tau</namePart>
            <namePart type="family">Yih</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Punta Cana, Dominican Republic</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical understanding is required. Some possible reasons can be the tokenizers and pre-training objectives which are not specifically designed to learn and preserve numeracy. Here we investigate the ability of text-to-text transfer learning model (T5), which has outperformed its predecessors in the conventional NLP tasks, to learn numeracy. We consider four numeracy tasks: numeration, magnitude order prediction, finding minimum and maximum in a series, and sorting. We find that, although T5 models perform reasonably well in the interpolation setting, they struggle considerably in the extrapolation setting across all four tasks.</abstract>
    <identifier type="citekey">pal-baral-2021-investigating-numeracy</identifier>
    <identifier type="doi">10.18653/v1/2021.findings-emnlp.265</identifier>
    <location>
        <url>https://aclanthology.org/2021.findings-emnlp.265</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>3095</start>
            <end>3101</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model
%A Pal, Kuntal Kumar
%A Baral, Chitta
%Y Moens, Marie-Francine
%Y Huang, Xuanjing
%Y Specia, Lucia
%Y Yih, Scott Wen-tau
%S Findings of the Association for Computational Linguistics: EMNLP 2021
%D 2021
%8 November
%I Association for Computational Linguistics
%C Punta Cana, Dominican Republic
%F pal-baral-2021-investigating-numeracy
%X The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical understanding is required. Some possible reasons can be the tokenizers and pre-training objectives which are not specifically designed to learn and preserve numeracy. Here we investigate the ability of text-to-text transfer learning model (T5), which has outperformed its predecessors in the conventional NLP tasks, to learn numeracy. We consider four numeracy tasks: numeration, magnitude order prediction, finding minimum and maximum in a series, and sorting. We find that, although T5 models perform reasonably well in the interpolation setting, they struggle considerably in the extrapolation setting across all four tasks.
%R 10.18653/v1/2021.findings-emnlp.265
%U https://aclanthology.org/2021.findings-emnlp.265
%U https://doi.org/10.18653/v1/2021.findings-emnlp.265
%P 3095-3101

Markdown (Informal)

[Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model](https://aclanthology.org/2021.findings-emnlp.265) (Pal & Baral, Findings 2021)

• Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model (Pal & Baral, Findings 2021)

ACL

• Kuntal Kumar Pal and Chitta Baral. 2021. Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3095–3101, Punta Cana, Dominican Republic. Association for Computational Linguistics.