<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yoon-etal-2019-computer">
    <titleInfo>
        <title>Computer Assisted Annotation of Tension Development in TED Talks through Crowdsourcing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Seungwon</namePart>
        <namePart type="family">Yoon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Wonsuk</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jong</namePart>
        <namePart type="family">Park</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Silviu</namePart>
            <namePart type="family">Paun</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Dirk</namePart>
            <namePart type="family">Hovy</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hong Kong</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We propose a method of machine-assisted annotation for the identification of tension development, annotating whether the tension is increasing, decreasing, or staying unchanged. We use a neural network based prediction model, whose predicted results are given to the annotators as initial values for the options that they are asked to choose. By presenting such initial values to the annotators, the annotation task becomes an evaluation task where the annotators inspect whether or not the predicted results are correct. To demonstrate the effectiveness of our method, we performed the annotation task in both in-house and crowdsourced environments. For the crowdsourced environment, we compared the annotation results with and without our method of machine-assisted annotation. We find that the results with our method showed a higher agreement to the gold standard than those without, though our method had little effect at reducing the time for annotation. Our codes for the experiment are made publicly available.</abstract>
    <identifier type="citekey">yoon-etal-2019-computer</identifier>
    <identifier type="doi">10.18653/v1/D19-5906</identifier>
    <location>
        <url>https://aclanthology.org/D19-5906</url>
    </location>
    <part>
        <date>2019-11</date>
        <extent unit="page">
            <start>39</start>
            <end>47</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Computer Assisted Annotation of Tension Development in TED Talks through Crowdsourcing
%A Yoon, Seungwon
%A Yang, Wonsuk
%A Park, Jong
%Y Paun, Silviu
%Y Hovy, Dirk
%S Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP
%D 2019
%8 November
%I Association for Computational Linguistics
%C Hong Kong
%F yoon-etal-2019-computer
%X We propose a method of machine-assisted annotation for the identification of tension development, annotating whether the tension is increasing, decreasing, or staying unchanged. We use a neural network based prediction model, whose predicted results are given to the annotators as initial values for the options that they are asked to choose. By presenting such initial values to the annotators, the annotation task becomes an evaluation task where the annotators inspect whether or not the predicted results are correct. To demonstrate the effectiveness of our method, we performed the annotation task in both in-house and crowdsourced environments. For the crowdsourced environment, we compared the annotation results with and without our method of machine-assisted annotation. We find that the results with our method showed a higher agreement to the gold standard than those without, though our method had little effect at reducing the time for annotation. Our codes for the experiment are made publicly available.
%R 10.18653/v1/D19-5906
%U https://aclanthology.org/D19-5906
%U https://doi.org/10.18653/v1/D19-5906
%P 39-47

Markdown (Informal)

[Computer Assisted Annotation of Tension Development in TED Talks through Crowdsourcing](https://aclanthology.org/D19-5906) (Yoon et al., 2019)

• Computer Assisted Annotation of Tension Development in TED Talks through Crowdsourcing (Yoon et al., 2019)

ACL

• Seungwon Yoon, Wonsuk Yang, and Jong Park. 2019. Computer Assisted Annotation of Tension Development in TED Talks through Crowdsourcing. In Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP, pages 39–47, Hong Kong. Association for Computational Linguistics.