<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="griciute-etal-2022-cusp">
    <titleInfo>
        <title>On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Bernadeta</namePart>
        <namePart type="family">Griciūtė</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marc</namePart>
        <namePart type="family">Tanti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lucia</namePart>
        <namePart type="family">Donatelli</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Figurative Language Processing (FLP)</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates (Hybrid)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Utterly creative texts can sometimes be difficult to understand, balancing on the edge of comprehensibility. However, good language skills and common sense allow advanced language users both to interpret creative texts and to reject some linguistic input as nonsense. The goal of this paper is to evaluate whether the current language models are also able to make the distinction between a creative language use and nonsense. To test this, we have computed mean rank and pseudo-log-likelihood score (PLL) of metaphorical and nonsensical sentences, and fine-tuned several pretrained models (BERT, RoBERTa) for binary classification between the two categories. There was a significant difference in the mean ranks and PPL scores of the categories, and the classifier reached around 85.5% accuracy. The results raise further questions on what could have let to such satisfactory performance.</abstract>
    <identifier type="citekey">griciute-etal-2022-cusp</identifier>
    <location>
        <url>https://aclanthology.org/2022.flp-1.25</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>173</start>
            <end>177</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense?
%A Griciūtė, Bernadeta
%A Tanti, Marc
%A Donatelli, Lucia
%S Proceedings of the 3rd Workshop on Figurative Language Processing (FLP)
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates (Hybrid)
%F griciute-etal-2022-cusp
%X Utterly creative texts can sometimes be difficult to understand, balancing on the edge of comprehensibility. However, good language skills and common sense allow advanced language users both to interpret creative texts and to reject some linguistic input as nonsense. The goal of this paper is to evaluate whether the current language models are also able to make the distinction between a creative language use and nonsense. To test this, we have computed mean rank and pseudo-log-likelihood score (PLL) of metaphorical and nonsensical sentences, and fine-tuned several pretrained models (BERT, RoBERTa) for binary classification between the two categories. There was a significant difference in the mean ranks and PPL scores of the categories, and the classifier reached around 85.5% accuracy. The results raise further questions on what could have let to such satisfactory performance.
%U https://aclanthology.org/2022.flp-1.25
%P 173-177

Markdown (Informal)

[On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense?](https://aclanthology.org/2022.flp-1.25) (Griciūtė et al., FLP 2022)

• On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense? (Griciūtė et al., FLP 2022)

ACL

• Bernadeta Griciūtė, Marc Tanti, and Lucia Donatelli. 2022. On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense?. In Proceedings of the 3rd Workshop on Figurative Language Processing (FLP), pages 173–177, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.