<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="elazar-etal-2020-extraordinary">
    <titleInfo>
        <title>The Extraordinary Failure of Complement Coercion Crowdsourcing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yanai</namePart>
        <namePart type="family">Elazar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Victoria</namePart>
        <namePart type="family">Basmov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shauli</namePart>
        <namePart type="family">Ravfogel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yoav</namePart>
        <namePart type="family">Goldberg</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Reut</namePart>
        <namePart type="family">Tsarfaty</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the First Workshop on Insights from Negative Results in NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">João</namePart>
            <namePart type="family">Sedoc</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rumshisky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Crowdsourcing has eased and scaled up the collection of linguistic annotation in recent years. In this work, we follow known methodologies of collecting labeled data for the complement coercion phenomenon. These are constructions with an implied action — e.g., “I started a new book I bought last week”, where the implied action is reading. We aim to collect annotated data for this phenomenon by reducing it to either of two known tasks: Explicit Completion and Natural Language Inference. However, in both cases, crowdsourcing resulted in low agreement scores, even though we followed the same methodologies as in previous work. Why does the same process fail to yield high agreement scores? We specify our modeling schemes, highlight the differences with previous work and provide some insights about the task and possible explanations for the failure. We conclude that specific phenomena require tailored solutions, not only in specialized algorithms, but also in data collection methods.</abstract>
    <identifier type="citekey">elazar-etal-2020-extraordinary</identifier>
    <identifier type="doi">10.18653/v1/2020.insights-1.17</identifier>
    <location>
        <url>https://aclanthology.org/2020.insights-1.17</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>106</start>
            <end>116</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Extraordinary Failure of Complement Coercion Crowdsourcing
%A Elazar, Yanai
%A Basmov, Victoria
%A Ravfogel, Shauli
%A Goldberg, Yoav
%A Tsarfaty, Reut
%Y Rogers, Anna
%Y Sedoc, João
%Y Rumshisky, Anna
%S Proceedings of the First Workshop on Insights from Negative Results in NLP
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F elazar-etal-2020-extraordinary
%X Crowdsourcing has eased and scaled up the collection of linguistic annotation in recent years. In this work, we follow known methodologies of collecting labeled data for the complement coercion phenomenon. These are constructions with an implied action — e.g., “I started a new book I bought last week”, where the implied action is reading. We aim to collect annotated data for this phenomenon by reducing it to either of two known tasks: Explicit Completion and Natural Language Inference. However, in both cases, crowdsourcing resulted in low agreement scores, even though we followed the same methodologies as in previous work. Why does the same process fail to yield high agreement scores? We specify our modeling schemes, highlight the differences with previous work and provide some insights about the task and possible explanations for the failure. We conclude that specific phenomena require tailored solutions, not only in specialized algorithms, but also in data collection methods.
%R 10.18653/v1/2020.insights-1.17
%U https://aclanthology.org/2020.insights-1.17
%U https://doi.org/10.18653/v1/2020.insights-1.17
%P 106-116

Markdown (Informal)

[The Extraordinary Failure of Complement Coercion Crowdsourcing](https://aclanthology.org/2020.insights-1.17) (Elazar et al., insights 2020)

• The Extraordinary Failure of Complement Coercion Crowdsourcing (Elazar et al., insights 2020)

ACL

• Yanai Elazar, Victoria Basmov, Shauli Ravfogel, Yoav Goldberg, and Reut Tsarfaty. 2020. The Extraordinary Failure of Complement Coercion Crowdsourcing. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pages 106–116, Online. Association for Computational Linguistics.