<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2022-systematicity">
    <titleInfo>
        <title>Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Siyan</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Riley</namePart>
        <namePart type="family">Carlson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Christopher</namePart>
        <namePart type="family">Potts</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Levin et al. (2019) show experimentally that the interpretations of novel English noun compounds (e.g., stew skillet), while not fully compositional, are highly predictable based on whether the modifier and head refer to artifacts or natural kinds. Is the large language model GPT-3 governed by the same interpretive principles? To address this question, we first compare Levin et al.’s experimental data with GPT-3 generations, finding a high degree of similarity. However, this evidence is consistent with GPT-3 reasoning only about specific lexical items rather than the more abstract conceptual categories of Levin et al.’s theory. To probe more deeply, we construct prompts that require the relevant kind of conceptual reasoning. Here, we fail to find convincing evidence that GPT-3 is reasoning about more than just individual lexical items. These results highlight the importance of controlling for low-level distributional regularities when assessing whether a large language model latently encodes a deeper theory.</abstract>
    <identifier type="citekey">li-etal-2022-systematicity</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-emnlp.50</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-emnlp.50</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>717</start>
            <end>728</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds
%A Li, Siyan
%A Carlson, Riley
%A Potts, Christopher
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Findings of the Association for Computational Linguistics: EMNLP 2022
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F li-etal-2022-systematicity
%X Levin et al. (2019) show experimentally that the interpretations of novel English noun compounds (e.g., stew skillet), while not fully compositional, are highly predictable based on whether the modifier and head refer to artifacts or natural kinds. Is the large language model GPT-3 governed by the same interpretive principles? To address this question, we first compare Levin et al.’s experimental data with GPT-3 generations, finding a high degree of similarity. However, this evidence is consistent with GPT-3 reasoning only about specific lexical items rather than the more abstract conceptual categories of Levin et al.’s theory. To probe more deeply, we construct prompts that require the relevant kind of conceptual reasoning. Here, we fail to find convincing evidence that GPT-3 is reasoning about more than just individual lexical items. These results highlight the importance of controlling for low-level distributional regularities when assessing whether a large language model latently encodes a deeper theory.
%R 10.18653/v1/2022.findings-emnlp.50
%U https://aclanthology.org/2022.findings-emnlp.50
%U https://doi.org/10.18653/v1/2022.findings-emnlp.50
%P 717-728

Markdown (Informal)

[Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds](https://aclanthology.org/2022.findings-emnlp.50) (Li et al., Findings 2022)

• Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds (Li et al., Findings 2022)

ACL

• Siyan Li, Riley Carlson, and Christopher Potts. 2022. Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 717–728, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.