<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wang-2026-probing">
    <titleInfo>
        <title>Probing Pretraining “Quality”: Sociolinguistic Mobility as a Lens on Web Corpus Filters</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jessica</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 13th Web-as-Corpus Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Pedro</namePart>
            <namePart type="family">Ortiz Suarez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laurie</namePart>
            <namePart type="family">Burchell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nikola</namePart>
            <namePart type="family">Ljubešić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yves</namePart>
            <namePart type="family">Scherrer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Veronika</namePart>
            <namePart type="family">Laippala</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vuk</namePart>
            <namePart type="family">Dinić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Thom</namePart>
            <namePart type="family">Vaughan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-481-1</identifier>
    </relatedItem>
    <abstract>Data selection algorithms for large language model (LLM) pretraining are designed to mine ‘quality’ from the web, but that notion is murky at best. This study explores whether sociolinguistic mobility – realized through language use, in context – offers a lens for understanding what such algorithms actually select. As a case study, the FineWeb pretraining pipeline is examined for how it reshapes two tangible axes of language use: the geographical provenance and register composition of texts drawn from ICE and GloWbE.\par FineWeb rejection rates are highest, and fairly uniform across provenance groups, for web-native and high-context registers (90.9% and 99.2%, respectively) but within medium-context registers, association with the United States is moderately protective: the 7% rejection rate is well below the medium-context average of 38%, an effect corroborated by NPMI. Unlike prior work on learned, classifier-based filters and US-internal sociolect bias, this study finds comparable effects in a deterministic pipeline within the global spread of English. As pretraining data selection becomes increasingly opaque, these results suggest that algorithmic ‘quality’ does not represent a real-world range of English use manifested in the web-as-corpus – a gap made visible by tangible sociolinguistic dimensions.</abstract>
    <identifier type="citekey">wang-2026-probing</identifier>
    <location>
        <url>https://aclanthology.org/2026.wac-1.28/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>320</start>
            <end>330</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Probing Pretraining “Quality”: Sociolinguistic Mobility as a Lens on Web Corpus Filters
%A Wang, Jessica
%Y Ortiz Suarez, Pedro
%Y Burchell, Laurie
%Y Ljubešić, Nikola
%Y Scherrer, Yves
%Y Laippala, Veronika
%Y Dinić, Vuk
%Y Vaughan, Thom
%S Proceedings of the 13th Web-as-Corpus Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-481-1
%F wang-2026-probing
%X Data selection algorithms for large language model (LLM) pretraining are designed to mine ‘quality’ from the web, but that notion is murky at best. This study explores whether sociolinguistic mobility – realized through language use, in context – offers a lens for understanding what such algorithms actually select. As a case study, the FineWeb pretraining pipeline is examined for how it reshapes two tangible axes of language use: the geographical provenance and register composition of texts drawn from ICE and GloWbE.\par FineWeb rejection rates are highest, and fairly uniform across provenance groups, for web-native and high-context registers (90.9% and 99.2%, respectively) but within medium-context registers, association with the United States is moderately protective: the 7% rejection rate is well below the medium-context average of 38%, an effect corroborated by NPMI. Unlike prior work on learned, classifier-based filters and US-internal sociolect bias, this study finds comparable effects in a deterministic pipeline within the global spread of English. As pretraining data selection becomes increasingly opaque, these results suggest that algorithmic ‘quality’ does not represent a real-world range of English use manifested in the web-as-corpus – a gap made visible by tangible sociolinguistic dimensions.
%U https://aclanthology.org/2026.wac-1.28/
%P 320-330

Markdown (Informal)

[Probing Pretraining “Quality”: Sociolinguistic Mobility as a Lens on Web Corpus Filters](https://aclanthology.org/2026.wac-1.28/) (Wang, WAC 2026)

• Probing Pretraining “Quality”: Sociolinguistic Mobility as a Lens on Web Corpus Filters (Wang, WAC 2026)

ACL

• Jessica Wang. 2026. Probing Pretraining “Quality”: Sociolinguistic Mobility as a Lens on Web Corpus Filters. In Proceedings of the 13th Web-as-Corpus Workshop, pages 320–330, Budapest, Hungary. Association for Computational Linguistics.