<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tran-kretchmar-2024-towards">
    <titleInfo>
        <title>Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Son</namePart>
        <namePart type="given">Quoc</namePart>
        <namePart type="family">Tran</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Matt</namePart>
        <namePart type="family">Kretchmar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper proposes a novel training method to improve the robustness of Extractive Question Answering (EQA) models. Previous research has shown that existing models, when trained on EQA datasets that include unanswerable questions, demonstrate a significant lack of robustness against distribution shifts and adversarial attacks. Despite this, the inclusion of unanswerable questions in EQA training datasets is essential for ensuring real-world reliability. Our proposed training method includes a novel loss function for the EQA problem and challenges an implicit assumption present in numerous EQA datasets. Models trained with our method maintain in-domain performance while achieving a notable improvement on out-of-domain datasets. This results in an overall F1 score improvement of 5.7 across all testing sets. Furthermore, our models exhibit significantly enhanced robustness against two types of adversarial attacks, with a performance decrease of only about one-third compared to the default models.</abstract>
    <identifier type="citekey">tran-kretchmar-2024-towards</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-emnlp.121</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-emnlp.121/</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>2222</start>
            <end>2236</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology
%A Tran, Son Quoc
%A Kretchmar, Matt
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Findings of the Association for Computational Linguistics: EMNLP 2024
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F tran-kretchmar-2024-towards
%X This paper proposes a novel training method to improve the robustness of Extractive Question Answering (EQA) models. Previous research has shown that existing models, when trained on EQA datasets that include unanswerable questions, demonstrate a significant lack of robustness against distribution shifts and adversarial attacks. Despite this, the inclusion of unanswerable questions in EQA training datasets is essential for ensuring real-world reliability. Our proposed training method includes a novel loss function for the EQA problem and challenges an implicit assumption present in numerous EQA datasets. Models trained with our method maintain in-domain performance while achieving a notable improvement on out-of-domain datasets. This results in an overall F1 score improvement of 5.7 across all testing sets. Furthermore, our models exhibit significantly enhanced robustness against two types of adversarial attacks, with a performance decrease of only about one-third compared to the default models.
%R 10.18653/v1/2024.findings-emnlp.121
%U https://aclanthology.org/2024.findings-emnlp.121/
%U https://doi.org/10.18653/v1/2024.findings-emnlp.121
%P 2222-2236

Markdown (Informal)

[Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology](https://aclanthology.org/2024.findings-emnlp.121/) (Tran & Kretchmar, Findings 2024)

• Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology (Tran & Kretchmar, Findings 2024)

ACL

• Son Quoc Tran and Matt Kretchmar. 2024. Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2222–2236, Miami, Florida, USA. Association for Computational Linguistics.