<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="vu-iyyer-2019-encouraging">
    <titleInfo>
        <title>Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tu</namePart>
        <namePart type="family">Vu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mohit</namePart>
        <namePart type="family">Iyyer</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Korhonen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Traum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Màrquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Florence, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>While paragraph embedding models are remarkably effective for downstream classification tasks, what they learn and encode into a single vector remains opaque. In this paper, we investigate a state-of-the-art paragraph embedding method proposed by Zhang et al. (2017) and discover that it cannot reliably tell whether a given sentence occurs in the input paragraph or not. We formulate a sentence content task to probe for this basic linguistic property and find that even a much simpler bag-of-words method has no trouble solving it. This result motivates us to replace the reconstruction-based objective of Zhang et al. (2017) with our sentence content probe objective in a semi-supervised setting. Despite its simplicity, our objective improves over paragraph reconstruction in terms of (1) downstream classification accuracies on benchmark datasets, (2) faster training, and (3) better generalization ability.</abstract>
    <identifier type="citekey">vu-iyyer-2019-encouraging</identifier>
    <identifier type="doi">10.18653/v1/P19-1638</identifier>
    <location>
        <url>https://aclanthology.org/P19-1638</url>
    </location>
    <part>
        <date>2019-07</date>
        <extent unit="page">
            <start>6331</start>
            <end>6338</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification
%A Vu, Tu
%A Iyyer, Mohit
%Y Korhonen, Anna
%Y Traum, David
%Y Màrquez, Lluís
%S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
%D 2019
%8 July
%I Association for Computational Linguistics
%C Florence, Italy
%F vu-iyyer-2019-encouraging
%X While paragraph embedding models are remarkably effective for downstream classification tasks, what they learn and encode into a single vector remains opaque. In this paper, we investigate a state-of-the-art paragraph embedding method proposed by Zhang et al. (2017) and discover that it cannot reliably tell whether a given sentence occurs in the input paragraph or not. We formulate a sentence content task to probe for this basic linguistic property and find that even a much simpler bag-of-words method has no trouble solving it. This result motivates us to replace the reconstruction-based objective of Zhang et al. (2017) with our sentence content probe objective in a semi-supervised setting. Despite its simplicity, our objective improves over paragraph reconstruction in terms of (1) downstream classification accuracies on benchmark datasets, (2) faster training, and (3) better generalization ability.
%R 10.18653/v1/P19-1638
%U https://aclanthology.org/P19-1638
%U https://doi.org/10.18653/v1/P19-1638
%P 6331-6338

Markdown (Informal)

[Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification](https://aclanthology.org/P19-1638) (Vu & Iyyer, ACL 2019)

• Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification (Vu & Iyyer, ACL 2019)

ACL

• Tu Vu and Mohit Iyyer. 2019. Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6331–6338, Florence, Italy. Association for Computational Linguistics.