<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yung-etal-2019-crowdsourcing">
    <titleInfo>
        <title>Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Frances</namePart>
        <namePart type="family">Yung</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vera</namePart>
        <namePart type="family">Demberg</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Merel</namePart>
        <namePart type="family">Scholman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 13th Linguistic Annotation Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Annemarie</namePart>
            <namePart type="family">Friedrich</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Deniz</namePart>
            <namePart type="family">Zeyrek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jet</namePart>
            <namePart type="family">Hoek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Florence, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The perspective of being able to crowd-source coherence relations bears the promise of acquiring annotations for new texts quickly, which could then increase the size and variety of discourse-annotated corpora. It would also open the avenue to answering new research questions: Collecting annotations from a larger number of individuals per instance would allow to investigate the distribution of inferred relations, and to study individual differences in coherence relation interpretation. However, annotating coherence relations with untrained workers is not trivial. We here propose a novel two-step annotation procedure, which extends an earlier method by Scholman and Demberg (2017a). In our approach, coherence relation labels are inferred from connectives that workers insert into the text. We show that the proposed method leads to replicable coherence annotations, and analyse the agreement between the obtained relation labels and annotations from PDTB and RSTDT on the same texts.</abstract>
    <identifier type="citekey">yung-etal-2019-crowdsourcing</identifier>
    <identifier type="doi">10.18653/v1/W19-4003</identifier>
    <location>
        <url>https://aclanthology.org/W19-4003</url>
    </location>
    <part>
        <date>2019-08</date>
        <extent unit="page">
            <start>16</start>
            <end>25</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task
%A Yung, Frances
%A Demberg, Vera
%A Scholman, Merel
%Y Friedrich, Annemarie
%Y Zeyrek, Deniz
%Y Hoek, Jet
%S Proceedings of the 13th Linguistic Annotation Workshop
%D 2019
%8 August
%I Association for Computational Linguistics
%C Florence, Italy
%F yung-etal-2019-crowdsourcing
%X The perspective of being able to crowd-source coherence relations bears the promise of acquiring annotations for new texts quickly, which could then increase the size and variety of discourse-annotated corpora. It would also open the avenue to answering new research questions: Collecting annotations from a larger number of individuals per instance would allow to investigate the distribution of inferred relations, and to study individual differences in coherence relation interpretation. However, annotating coherence relations with untrained workers is not trivial. We here propose a novel two-step annotation procedure, which extends an earlier method by Scholman and Demberg (2017a). In our approach, coherence relation labels are inferred from connectives that workers insert into the text. We show that the proposed method leads to replicable coherence annotations, and analyse the agreement between the obtained relation labels and annotations from PDTB and RSTDT on the same texts.
%R 10.18653/v1/W19-4003
%U https://aclanthology.org/W19-4003
%U https://doi.org/10.18653/v1/W19-4003
%P 16-25

Markdown (Informal)

[Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task](https://aclanthology.org/W19-4003) (Yung et al., LAW 2019)

• Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task (Yung et al., LAW 2019)

ACL

• Frances Yung, Vera Demberg, and Merel Scholman. 2019. Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task. In Proceedings of the 13th Linguistic Annotation Workshop, pages 16–25, Florence, Italy. Association for Computational Linguistics.