<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="schofield-etal-2017-quantifying">
    <titleInfo>
        <title>Quantifying the Effects of Text Duplication on Semantic Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Alexandra</namePart>
        <namePart type="family">Schofield</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Laure</namePart>
        <namePart type="family">Thompson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">David</namePart>
        <namePart type="family">Mimno</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Martha</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rebecca</namePart>
            <namePart type="family">Hwa</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sebastian</namePart>
            <namePart type="family">Riedel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Duplicate documents are a pervasive problem in text datasets and can have a strong effect on unsupervised models. Methods to remove duplicate texts are typically heuristic or very expensive, so it is vital to know when and why they are needed. We measure the sensitivity of two latent semantic methods to the presence of different levels of document repetition. By artificially creating different forms of duplicate text we confirm several hypotheses about how repeated text impacts models. While a small amount of duplication is tolerable, substantial over-representation of subsets of the text may overwhelm meaningful topical patterns.</abstract>
    <identifier type="citekey">schofield-etal-2017-quantifying</identifier>
    <identifier type="doi">10.18653/v1/D17-1290</identifier>
    <location>
        <url>https://aclanthology.org/D17-1290</url>
    </location>
    <part>
        <date>2017-09</date>
        <extent unit="page">
            <start>2737</start>
            <end>2747</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Quantifying the Effects of Text Duplication on Semantic Models
%A Schofield, Alexandra
%A Thompson, Laure
%A Mimno, David
%Y Palmer, Martha
%Y Hwa, Rebecca
%Y Riedel, Sebastian
%S Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
%D 2017
%8 September
%I Association for Computational Linguistics
%C Copenhagen, Denmark
%F schofield-etal-2017-quantifying
%X Duplicate documents are a pervasive problem in text datasets and can have a strong effect on unsupervised models. Methods to remove duplicate texts are typically heuristic or very expensive, so it is vital to know when and why they are needed. We measure the sensitivity of two latent semantic methods to the presence of different levels of document repetition. By artificially creating different forms of duplicate text we confirm several hypotheses about how repeated text impacts models. While a small amount of duplication is tolerable, substantial over-representation of subsets of the text may overwhelm meaningful topical patterns.
%R 10.18653/v1/D17-1290
%U https://aclanthology.org/D17-1290
%U https://doi.org/10.18653/v1/D17-1290
%P 2737-2747

Markdown (Informal)

[Quantifying the Effects of Text Duplication on Semantic Models](https://aclanthology.org/D17-1290) (Schofield et al., EMNLP 2017)

• Quantifying the Effects of Text Duplication on Semantic Models (Schofield et al., EMNLP 2017)

ACL

• Alexandra Schofield, Laure Thompson, and David Mimno. 2017. Quantifying the Effects of Text Duplication on Semantic Models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2737–2747, Copenhagen, Denmark. Association for Computational Linguistics.