<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pandia-etal-2021-pragmatic">
    <titleInfo>
        <title>Pragmatic competence of pre-trained language models through the lens of discourse connectives</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lalchand</namePart>
        <namePart type="family">Pandia</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yan</namePart>
        <namePart type="family">Cong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Allyson</namePart>
        <namePart type="family">Ettinger</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 25th Conference on Computational Natural Language Learning</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Arianna</namePart>
            <namePart type="family">Bisazza</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Omri</namePart>
            <namePart type="family">Abend</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>As pre-trained language models (LMs) continue to dominate NLP, it is increasingly important that we understand the depth of language capabilities in these models. In this paper, we target pre-trained LMs’ competence in pragmatics, with a focus on pragmatics relating to discourse connectives. We formulate cloze-style tests using a combination of naturally-occurring data and controlled inputs drawn from psycholinguistics. We focus on testing models’ ability to use pragmatic cues to predict discourse connectives, models’ ability to understand implicatures relating to connectives, and the extent to which models show humanlike preferences regarding temporal dynamics of connectives. We find that although models predict connectives reasonably well in the context of naturally-occurring data, when we control contexts to isolate high-level pragmatic cues, model sensitivity is much lower. Models also do not show substantial humanlike temporal preferences. Overall, the findings suggest that at present, dominant pre-training paradigms do not result in substantial pragmatic competence in our models.</abstract>
    <identifier type="citekey">pandia-etal-2021-pragmatic</identifier>
    <identifier type="doi">10.18653/v1/2021.conll-1.29</identifier>
    <location>
        <url>https://aclanthology.org/2021.conll-1.29</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>367</start>
            <end>379</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Pragmatic competence of pre-trained language models through the lens of discourse connectives
%A Pandia, Lalchand
%A Cong, Yan
%A Ettinger, Allyson
%Y Bisazza, Arianna
%Y Abend, Omri
%S Proceedings of the 25th Conference on Computational Natural Language Learning
%D 2021
%8 November
%I Association for Computational Linguistics
%C Online
%F pandia-etal-2021-pragmatic
%X As pre-trained language models (LMs) continue to dominate NLP, it is increasingly important that we understand the depth of language capabilities in these models. In this paper, we target pre-trained LMs’ competence in pragmatics, with a focus on pragmatics relating to discourse connectives. We formulate cloze-style tests using a combination of naturally-occurring data and controlled inputs drawn from psycholinguistics. We focus on testing models’ ability to use pragmatic cues to predict discourse connectives, models’ ability to understand implicatures relating to connectives, and the extent to which models show humanlike preferences regarding temporal dynamics of connectives. We find that although models predict connectives reasonably well in the context of naturally-occurring data, when we control contexts to isolate high-level pragmatic cues, model sensitivity is much lower. Models also do not show substantial humanlike temporal preferences. Overall, the findings suggest that at present, dominant pre-training paradigms do not result in substantial pragmatic competence in our models.
%R 10.18653/v1/2021.conll-1.29
%U https://aclanthology.org/2021.conll-1.29
%U https://doi.org/10.18653/v1/2021.conll-1.29
%P 367-379

Markdown (Informal)

[Pragmatic competence of pre-trained language models through the lens of discourse connectives](https://aclanthology.org/2021.conll-1.29) (Pandia et al., CoNLL 2021)

• Pragmatic competence of pre-trained language models through the lens of discourse connectives (Pandia et al., CoNLL 2021)

ACL

• Lalchand Pandia, Yan Cong, and Allyson Ettinger. 2021. Pragmatic competence of pre-trained language models through the lens of discourse connectives. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 367–379, Online. Association for Computational Linguistics.