<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nielsen-etal-2023-spelling">
    <titleInfo>
        <title>Spelling convention sensitivity in neural language models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Elizabeth</namePart>
        <namePart type="family">Nielsen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Christo</namePart>
        <namePart type="family">Kirov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Brian</namePart>
        <namePart type="family">Roark</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We examine whether large neural language models, trained on very large collections of varied English text, learn the potentially long-distance dependency of British versus American spelling conventions, i.e., whether spelling is consistently one or the other within model-generated strings. In contrast to long-distance dependencies in non-surface underlying structure (e.g., syntax), spelling consistency is easier to measure both in LMs and the text corpora used to train them, which can provide additional insight into certain observed model behaviors. Using a set of probe words unique to either British or American English, we first establish that training corpora exhibit substantial (though not total) consistency. A large T5 language model does appear to internalize this consistency, though only with respect to observed lexical items (not nonce words with British/American spelling patterns). We further experiment with correcting for biases in the training data by fine-tuning T5 on synthetic data that has been debiased, and find that finetuned T5 remains only somewhat sensitive to spelling consistency. Further experiments show GPT2 to be similarly limited.</abstract>
    <identifier type="citekey">nielsen-etal-2023-spelling</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-eacl.98</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-eacl.98</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>1334</start>
            <end>1346</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Spelling convention sensitivity in neural language models
%A Nielsen, Elizabeth
%A Kirov, Christo
%A Roark, Brian
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Findings of the Association for Computational Linguistics: EACL 2023
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F nielsen-etal-2023-spelling
%X We examine whether large neural language models, trained on very large collections of varied English text, learn the potentially long-distance dependency of British versus American spelling conventions, i.e., whether spelling is consistently one or the other within model-generated strings. In contrast to long-distance dependencies in non-surface underlying structure (e.g., syntax), spelling consistency is easier to measure both in LMs and the text corpora used to train them, which can provide additional insight into certain observed model behaviors. Using a set of probe words unique to either British or American English, we first establish that training corpora exhibit substantial (though not total) consistency. A large T5 language model does appear to internalize this consistency, though only with respect to observed lexical items (not nonce words with British/American spelling patterns). We further experiment with correcting for biases in the training data by fine-tuning T5 on synthetic data that has been debiased, and find that finetuned T5 remains only somewhat sensitive to spelling consistency. Further experiments show GPT2 to be similarly limited.
%R 10.18653/v1/2023.findings-eacl.98
%U https://aclanthology.org/2023.findings-eacl.98
%U https://doi.org/10.18653/v1/2023.findings-eacl.98
%P 1334-1346

Markdown (Informal)

[Spelling convention sensitivity in neural language models](https://aclanthology.org/2023.findings-eacl.98) (Nielsen et al., Findings 2023)

• Spelling convention sensitivity in neural language models (Nielsen et al., Findings 2023)

ACL

• Elizabeth Nielsen, Christo Kirov, and Brian Roark. 2023. Spelling convention sensitivity in neural language models. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1334–1346, Dubrovnik, Croatia. Association for Computational Linguistics.