<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="goyal-2026-crawl">
    <titleInfo>
        <title>The Crawl Gap: Where Endangered-Language Web Content Is Lost Before It Reaches a Corpus</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sharvin</namePart>
        <namePart type="family">Goyal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 13th Web-as-Corpus Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Pedro</namePart>
            <namePart type="family">Ortiz Suarez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laurie</namePart>
            <namePart type="family">Burchell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nikola</namePart>
            <namePart type="family">Ljubešić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yves</namePart>
            <namePart type="family">Scherrer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Veronika</namePart>
            <namePart type="family">Laippala</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vuk</namePart>
            <namePart type="family">Dinić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Thom</namePart>
            <namePart type="family">Vaughan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-481-1</identifier>
    </relatedItem>
    <abstract>Web-crawled corpora now claim that they have coverage of more than a thousand languages, but their subsets for endangered languages remain very small. Prior audits examine what is inside these corpora. We ask a question that complements this idea: what happens to the endangered-language web content that does exist? We curate and verify an inventory of 40 pages from active institutional websites (tribal language programs, community dictionaries, and learning portals) for 16 endangered North American languages from 7 families, and we attribute losses along the full pipeline: (S1) whether Common Crawl ever crawled the page, (S2) whether language identification (LID) recognizes the captured text, (S3) whether a standard quality-filter cascade can help retain it, and (S4) whether any of it reaches a released corpus (FineWeb-2, GlotCC). Most of the loss happens before corpus construction even begins. These websites present to crawlers almost entirely in English, much of the actual language text is served by client-rendered dictionary applications that expose no text to crawlers, 6 of 16 languages have no label in the standard LID model, and none of the 36 inventory domains contributes a single document to any released per-language subset. We release the inventory, the pipeline, and the per-stage attribution to support both crawl-side and community-side remedies.</abstract>
    <identifier type="citekey">goyal-2026-crawl</identifier>
    <location>
        <url>https://aclanthology.org/2026.wac-1.21/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>220</start>
            <end>223</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Crawl Gap: Where Endangered-Language Web Content Is Lost Before It Reaches a Corpus
%A Goyal, Sharvin
%Y Ortiz Suarez, Pedro
%Y Burchell, Laurie
%Y Ljubešić, Nikola
%Y Scherrer, Yves
%Y Laippala, Veronika
%Y Dinić, Vuk
%Y Vaughan, Thom
%S Proceedings of the 13th Web-as-Corpus Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-481-1
%F goyal-2026-crawl
%X Web-crawled corpora now claim that they have coverage of more than a thousand languages, but their subsets for endangered languages remain very small. Prior audits examine what is inside these corpora. We ask a question that complements this idea: what happens to the endangered-language web content that does exist? We curate and verify an inventory of 40 pages from active institutional websites (tribal language programs, community dictionaries, and learning portals) for 16 endangered North American languages from 7 families, and we attribute losses along the full pipeline: (S1) whether Common Crawl ever crawled the page, (S2) whether language identification (LID) recognizes the captured text, (S3) whether a standard quality-filter cascade can help retain it, and (S4) whether any of it reaches a released corpus (FineWeb-2, GlotCC). Most of the loss happens before corpus construction even begins. These websites present to crawlers almost entirely in English, much of the actual language text is served by client-rendered dictionary applications that expose no text to crawlers, 6 of 16 languages have no label in the standard LID model, and none of the 36 inventory domains contributes a single document to any released per-language subset. We release the inventory, the pipeline, and the per-stage attribution to support both crawl-side and community-side remedies.
%U https://aclanthology.org/2026.wac-1.21/
%P 220-223

Markdown (Informal)

[The Crawl Gap: Where Endangered-Language Web Content Is Lost Before It Reaches a Corpus](https://aclanthology.org/2026.wac-1.21/) (Goyal, WAC 2026)

• The Crawl Gap: Where Endangered-Language Web Content Is Lost Before It Reaches a Corpus (Goyal, WAC 2026)

ACL

• Sharvin Goyal. 2026. The Crawl Gap: Where Endangered-Language Web Content Is Lost Before It Reaches a Corpus. In Proceedings of the 13th Web-as-Corpus Workshop, pages 220–223, Budapest, Hungary. Association for Computational Linguistics.