<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="al-negheimish-etal-2021-numerical">
    <titleInfo>
        <title>Numerical reasoning in machine reading comprehension tasks: are we there yet?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hadeel</namePart>
        <namePart type="family">Al-Negheimish</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pranava</namePart>
        <namePart type="family">Madhyastha</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alessandra</namePart>
        <namePart type="family">Russo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marie-Francine</namePart>
            <namePart type="family">Moens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Xuanjing</namePart>
            <namePart type="family">Huang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lucia</namePart>
            <namePart type="family">Specia</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Scott</namePart>
            <namePart type="given">Wen-tau</namePart>
            <namePart type="family">Yih</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online and Punta Cana, Dominican Republic</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Numerical reasoning based machine reading comprehension is a task that involves reading comprehension along with using arithmetic operations such as addition, subtraction, sorting and counting. The DROP benchmark (Dua et al., 2019) is a recent dataset that has inspired the design of NLP models aimed at solving this task. The current standings of these models in the DROP leaderboard, over standard metrics, suggests that the models have achieved near-human performance. However, does this mean that these models have learned to reason? In this paper, we present a controlled study on some of the top-performing model architectures for the task of numerical reasoning. Our observations suggest that the standard metrics are incapable of measuring progress towards such tasks.</abstract>
    <identifier type="citekey">al-negheimish-etal-2021-numerical</identifier>
    <identifier type="doi">10.18653/v1/2021.emnlp-main.759</identifier>
    <location>
        <url>https://aclanthology.org/2021.emnlp-main.759</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>9643</start>
            <end>9649</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Numerical reasoning in machine reading comprehension tasks: are we there yet?
%A Al-Negheimish, Hadeel
%A Madhyastha, Pranava
%A Russo, Alessandra
%Y Moens, Marie-Francine
%Y Huang, Xuanjing
%Y Specia, Lucia
%Y Yih, Scott Wen-tau
%S Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
%D 2021
%8 November
%I Association for Computational Linguistics
%C Online and Punta Cana, Dominican Republic
%F al-negheimish-etal-2021-numerical
%X Numerical reasoning based machine reading comprehension is a task that involves reading comprehension along with using arithmetic operations such as addition, subtraction, sorting and counting. The DROP benchmark (Dua et al., 2019) is a recent dataset that has inspired the design of NLP models aimed at solving this task. The current standings of these models in the DROP leaderboard, over standard metrics, suggests that the models have achieved near-human performance. However, does this mean that these models have learned to reason? In this paper, we present a controlled study on some of the top-performing model architectures for the task of numerical reasoning. Our observations suggest that the standard metrics are incapable of measuring progress towards such tasks.
%R 10.18653/v1/2021.emnlp-main.759
%U https://aclanthology.org/2021.emnlp-main.759
%U https://doi.org/10.18653/v1/2021.emnlp-main.759
%P 9643-9649

Markdown (Informal)

[Numerical reasoning in machine reading comprehension tasks: are we there yet?](https://aclanthology.org/2021.emnlp-main.759) (Al-Negheimish et al., EMNLP 2021)

• Numerical reasoning in machine reading comprehension tasks: are we there yet? (Al-Negheimish et al., EMNLP 2021)

ACL

• Hadeel Al-Negheimish, Pranava Madhyastha, and Alessandra Russo. 2021. Numerical reasoning in machine reading comprehension tasks: are we there yet?. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9643–9649, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.