<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hou-smith-2023-detecting">
    <titleInfo>
        <title>Detecting Syntactic Change with Pre-trained Transformer Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Liwen</namePart>
        <namePart type="family">Hou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">David</namePart>
        <namePart type="family">Smith</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We investigate the ability of Transformer-based language models to find syntactic differences between the English of the early 1800s and that of the late 1900s. First, we show that a fine-tuned BERT model can distinguish between text from these two periods using syntactic information only; to show this, we employ a strategy to hide semantic information from the text. Second, we make further use of fine-tuned BERT models to identify specific instances of syntactic change and specific words for which a new part of speech was introduced. To do this, we employ an automatic part-of-speech (POS) tagger and use it to train corpora-specific taggers based only on BERT representations pretrained on different corpora. Notably, our methods of identifying specific candidates for syntactic change avoid using any automatic POS tagger on old text, where its performance may be unreliable; instead, our methods only use untagged old text together with tagged modern text. We examine samples and distributional properties of the model output to validate automatically identified cases of syntactic change. Finally, we use our techniques to confirm the historical rise of the progressive construction, a known example of syntactic change.</abstract>
    <identifier type="citekey">hou-smith-2023-detecting</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-emnlp.230</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-emnlp.230</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>3564</start>
            <end>3574</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Detecting Syntactic Change with Pre-trained Transformer Models
%A Hou, Liwen
%A Smith, David
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Findings of the Association for Computational Linguistics: EMNLP 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F hou-smith-2023-detecting
%X We investigate the ability of Transformer-based language models to find syntactic differences between the English of the early 1800s and that of the late 1900s. First, we show that a fine-tuned BERT model can distinguish between text from these two periods using syntactic information only; to show this, we employ a strategy to hide semantic information from the text. Second, we make further use of fine-tuned BERT models to identify specific instances of syntactic change and specific words for which a new part of speech was introduced. To do this, we employ an automatic part-of-speech (POS) tagger and use it to train corpora-specific taggers based only on BERT representations pretrained on different corpora. Notably, our methods of identifying specific candidates for syntactic change avoid using any automatic POS tagger on old text, where its performance may be unreliable; instead, our methods only use untagged old text together with tagged modern text. We examine samples and distributional properties of the model output to validate automatically identified cases of syntactic change. Finally, we use our techniques to confirm the historical rise of the progressive construction, a known example of syntactic change.
%R 10.18653/v1/2023.findings-emnlp.230
%U https://aclanthology.org/2023.findings-emnlp.230
%U https://doi.org/10.18653/v1/2023.findings-emnlp.230
%P 3564-3574

Markdown (Informal)

[Detecting Syntactic Change with Pre-trained Transformer Models](https://aclanthology.org/2023.findings-emnlp.230) (Hou & Smith, Findings 2023)

• Detecting Syntactic Change with Pre-trained Transformer Models (Hou & Smith, Findings 2023)

ACL

• Liwen Hou and David Smith. 2023. Detecting Syntactic Change with Pre-trained Transformer Models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3564–3574, Singapore. Association for Computational Linguistics.