<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gibson-etal-2008-identification">
    <titleInfo>
        <title>Identification of Duplicate News Stories in Web Pages</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">John</namePart>
        <namePart type="family">Gibson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ben</namePart>
        <namePart type="family">Wellner</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Susan</namePart>
        <namePart type="family">Lubar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2008-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 4th Web as Corpus Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Stefan</namePart>
            <namePart type="family">Evert</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Adam</namePart>
            <namePart type="family">Kilgarriff</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Serge</namePart>
            <namePart type="family">Sharoff</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>European Language Resources Association</publisher>
            <place>
                <placeTerm type="text">Marrakech, Morocco</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Identifying near duplicate documents is a challenge often faced in the field of information discovery. Unfortunately many algorithms that find near duplicate pairs of plain text documents perform poorly when used on web pages, where metadata, other extraneous information make that process much more difficult. If the content of the page (e.g., the body of a news article) can be extracted from the page, then the accuracy of the duplicate detection algorithms is greatly increased. Using machine learning techniques to identify the content portion of web pages, we achieve duplicate detection accuracy that is nearly identical to plain text, significantly better than simple heuristic approaches to content extraction. We performed these experiments on a small, but fully annotated corpus.</abstract>
    <identifier type="citekey">gibson-etal-2008-identification</identifier>
    <location>
        <url>https://aclanthology.org/2008.wac-1.5/</url>
    </location>
    <part>
        <date>2008-06</date>
        <extent unit="page">
            <start>26</start>
            <end>33</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Identification of Duplicate News Stories in Web Pages
%A Gibson, John
%A Wellner, Ben
%A Lubar, Susan
%Y Evert, Stefan
%Y Kilgarriff, Adam
%Y Sharoff, Serge
%S Proceedings of the 4th Web as Corpus Workshop
%D 2008
%8 June
%I European Language Resources Association
%C Marrakech, Morocco
%F gibson-etal-2008-identification
%X Identifying near duplicate documents is a challenge often faced in the field of information discovery. Unfortunately many algorithms that find near duplicate pairs of plain text documents perform poorly when used on web pages, where metadata, other extraneous information make that process much more difficult. If the content of the page (e.g., the body of a news article) can be extracted from the page, then the accuracy of the duplicate detection algorithms is greatly increased. Using machine learning techniques to identify the content portion of web pages, we achieve duplicate detection accuracy that is nearly identical to plain text, significantly better than simple heuristic approaches to content extraction. We performed these experiments on a small, but fully annotated corpus.
%U https://aclanthology.org/2008.wac-1.5/
%P 26-33

Markdown (Informal)

[Identification of Duplicate News Stories in Web Pages](https://aclanthology.org/2008.wac-1.5/) (Gibson et al., WAC 2008)

• Identification of Duplicate News Stories in Web Pages (Gibson et al., WAC 2008)

ACL

• John Gibson, Ben Wellner, and Susan Lubar. 2008. Identification of Duplicate News Stories in Web Pages. In Proceedings of the 4th Web as Corpus Workshop, pages 26–33, Marrakech, Morocco. European Language Resources Association.