<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yu-etal-2020-word">
    <titleInfo>
        <title>Word Frequency Does Not Predict Grammatical Knowledge in Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Charles</namePart>
        <namePart type="family">Yu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ryan</namePart>
        <namePart type="family">Sie</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nicolas</namePart>
        <namePart type="family">Tedeschi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Leon</namePart>
        <namePart type="family">Bergen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Bonnie</namePart>
            <namePart type="family">Webber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Trevor</namePart>
            <namePart type="family">Cohn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulan</namePart>
            <namePart type="family">He</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Neural language models learn, to varying degrees of accuracy, the grammatical properties of natural languages. In this work, we investigate whether there are systematic sources of variation in the language models’ accuracy. Focusing on subject-verb agreement and reflexive anaphora, we find that certain nouns are systematically understood better than others, an effect which is robust across grammatical tasks and different language models. Surprisingly, we find that across four orders of magnitude, corpus frequency is unrelated to a noun’s performance on grammatical tasks. Finally, we find that a novel noun’s grammatical properties can be few-shot learned from various types of training data. The results present a paradox: there should be less variation in grammatical performance than is actually observed.</abstract>
    <identifier type="citekey">yu-etal-2020-word</identifier>
    <identifier type="doi">10.18653/v1/2020.emnlp-main.331</identifier>
    <location>
        <url>https://aclanthology.org/2020.emnlp-main.331</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>4040</start>
            <end>4054</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Word Frequency Does Not Predict Grammatical Knowledge in Language Models
%A Yu, Charles
%A Sie, Ryan
%A Tedeschi, Nicolas
%A Bergen, Leon
%Y Webber, Bonnie
%Y Cohn, Trevor
%Y He, Yulan
%Y Liu, Yang
%S Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F yu-etal-2020-word
%X Neural language models learn, to varying degrees of accuracy, the grammatical properties of natural languages. In this work, we investigate whether there are systematic sources of variation in the language models’ accuracy. Focusing on subject-verb agreement and reflexive anaphora, we find that certain nouns are systematically understood better than others, an effect which is robust across grammatical tasks and different language models. Surprisingly, we find that across four orders of magnitude, corpus frequency is unrelated to a noun’s performance on grammatical tasks. Finally, we find that a novel noun’s grammatical properties can be few-shot learned from various types of training data. The results present a paradox: there should be less variation in grammatical performance than is actually observed.
%R 10.18653/v1/2020.emnlp-main.331
%U https://aclanthology.org/2020.emnlp-main.331
%U https://doi.org/10.18653/v1/2020.emnlp-main.331
%P 4040-4054

Markdown (Informal)

[Word Frequency Does Not Predict Grammatical Knowledge in Language Models](https://aclanthology.org/2020.emnlp-main.331) (Yu et al., EMNLP 2020)

• Word Frequency Does Not Predict Grammatical Knowledge in Language Models (Yu et al., EMNLP 2020)

ACL

• Charles Yu, Ryan Sie, Nicolas Tedeschi, and Leon Bergen. 2020. Word Frequency Does Not Predict Grammatical Knowledge in Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4040–4054, Online. Association for Computational Linguistics.