<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="miyakawa-etal-2024-llms">
    <titleInfo>
        <title>Do LLMs Agree with Humans on Emotional Associations to Nonsense Words?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yui</namePart>
        <namePart type="family">Miyakawa</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chihaya</namePart>
        <namePart type="family">Matsuhira</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hirotaka</namePart>
        <namePart type="family">Kato</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Takatsugu</namePart>
        <namePart type="family">Hirayama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Takahiro</namePart>
        <namePart type="family">Komamizu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ichiro</namePart>
        <namePart type="family">Ide</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Tatsuki</namePart>
            <namePart type="family">Kuribayashi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Giulia</namePart>
            <namePart type="family">Rambelli</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ece</namePart>
            <namePart type="family">Takmaz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Wicke</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yohei</namePart>
            <namePart type="family">Oseki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Understanding human perception of nonsense words is helpful to devise product and character names that match their characteristics. Previous studies have suggested the usefulness of Large Language Models (LLMs) for estimating such human perception, but they did not focus on its emotional aspects. Hence, this study aims to elucidate the relationship of emotions evoked by nonsense words between humans and LLMs. Using a representative LLM, GPT-4, we reproduce the procedure of an existing study to analyze evoked emotions of humans for nonsense words. A positive correlation of 0.40 was found between the emotion intensity scores reproduced by GPT-4 and those manually annotated by humans. Although the correlation is not very high, this demonstrates that GPT-4 may agree with humans on emotional associations to nonsense words. Considering that the previous study reported that the correlation among human annotators was about 0.68 on average and that between a regression model trained on the annotations for real words and humans was 0.17, GPT-4’s agreement with humans is notably strong.</abstract>
    <identifier type="citekey">miyakawa-etal-2024-llms</identifier>
    <location>
        <url>https://aclanthology.org/2024.cmcl-1.7</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>81</start>
            <end>85</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Do LLMs Agree with Humans on Emotional Associations to Nonsense Words?
%A Miyakawa, Yui
%A Matsuhira, Chihaya
%A Kato, Hirotaka
%A Hirayama, Takatsugu
%A Komamizu, Takahiro
%A Ide, Ichiro
%Y Kuribayashi, Tatsuki
%Y Rambelli, Giulia
%Y Takmaz, Ece
%Y Wicke, Philipp
%Y Oseki, Yohei
%S Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F miyakawa-etal-2024-llms
%X Understanding human perception of nonsense words is helpful to devise product and character names that match their characteristics. Previous studies have suggested the usefulness of Large Language Models (LLMs) for estimating such human perception, but they did not focus on its emotional aspects. Hence, this study aims to elucidate the relationship of emotions evoked by nonsense words between humans and LLMs. Using a representative LLM, GPT-4, we reproduce the procedure of an existing study to analyze evoked emotions of humans for nonsense words. A positive correlation of 0.40 was found between the emotion intensity scores reproduced by GPT-4 and those manually annotated by humans. Although the correlation is not very high, this demonstrates that GPT-4 may agree with humans on emotional associations to nonsense words. Considering that the previous study reported that the correlation among human annotators was about 0.68 on average and that between a regression model trained on the annotations for real words and humans was 0.17, GPT-4’s agreement with humans is notably strong.
%U https://aclanthology.org/2024.cmcl-1.7
%P 81-85

Markdown (Informal)

[Do LLMs Agree with Humans on Emotional Associations to Nonsense Words?](https://aclanthology.org/2024.cmcl-1.7) (Miyakawa et al., CMCL-WS 2024)

• Do LLMs Agree with Humans on Emotional Associations to Nonsense Words? (Miyakawa et al., CMCL-WS 2024)

ACL

• Yui Miyakawa, Chihaya Matsuhira, Hirotaka Kato, Takatsugu Hirayama, Takahiro Komamizu, and Ichiro Ide. 2024. Do LLMs Agree with Humans on Emotional Associations to Nonsense Words?. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 81–85, Bangkok, Thailand. Association for Computational Linguistics.