<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wallace-etal-2019-trick">
    <titleInfo>
        <title>Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Eric</namePart>
        <namePart type="family">Wallace</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pedro</namePart>
        <namePart type="family">Rodriguez</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shi</namePart>
        <namePart type="family">Feng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ikuya</namePart>
        <namePart type="family">Yamada</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jordan</namePart>
        <namePart type="family">Boyd-Graber</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Adversarial evaluation stress-tests a model’s understanding of natural language. Because past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human- in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human–computer matches: Although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.</abstract>
    <identifier type="citekey">wallace-etal-2019-trick</identifier>
    <identifier type="doi">10.1162/tacl_a_00279</identifier>
    <location>
        <url>https://aclanthology.org/Q19-1029</url>
    </location>
    <part>
        <date>2019</date>
        <detail type="volume"><number>7</number></detail>
        <extent unit="page">
            <start>387</start>
            <end>401</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering
%A Wallace, Eric
%A Rodriguez, Pedro
%A Feng, Shi
%A Yamada, Ikuya
%A Boyd-Graber, Jordan
%J Transactions of the Association for Computational Linguistics
%D 2019
%V 7
%I MIT Press
%C Cambridge, MA
%F wallace-etal-2019-trick
%X Adversarial evaluation stress-tests a model’s understanding of natural language. Because past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human- in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human–computer matches: Although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.
%R 10.1162/tacl_a_00279
%U https://aclanthology.org/Q19-1029
%U https://doi.org/10.1162/tacl_a_00279
%P 387-401

Markdown (Informal)

[Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering](https://aclanthology.org/Q19-1029) (Wallace et al., TACL 2019)

• Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering (Wallace et al., TACL 2019)

ACL

• Eric Wallace, Pedro Rodriguez, Shi Feng, Ikuya Yamada, and Jordan Boyd-Graber. 2019. Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering. Transactions of the Association for Computational Linguistics, 7:387–401.