<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chen-ding-2023-probing">
    <titleInfo>
        <title>Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Honghua</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nai</namePart>
        <namePart type="family">Ding</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large language models possess remarkable capacity for processing language, but it remains unclear whether these models can further generate creative content. The present study aims to investigate the creative thinking of large language models through a cognitive perspective. We utilize the divergent association task (DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. We compare the results across different models and decoding strategies. Our findings indicate that: (1) When using the greedy search strategy, GPT-4 outperforms 96% of humans, while GPT-3.5-turbo exceeds the average human level. (2) Stochastic sampling and temperature scaling are effective to obtain higher DAT scores for models except GPT-4, but face a trade-off between creativity and stability. These results imply that advanced large language models have divergent semantic associations, which is a fundamental process underlying creativity.</abstract>
    <identifier type="citekey">chen-ding-2023-probing</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-emnlp.858</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-emnlp.858</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>12881</start>
            <end>12888</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association?
%A Chen, Honghua
%A Ding, Nai
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Findings of the Association for Computational Linguistics: EMNLP 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F chen-ding-2023-probing
%X Large language models possess remarkable capacity for processing language, but it remains unclear whether these models can further generate creative content. The present study aims to investigate the creative thinking of large language models through a cognitive perspective. We utilize the divergent association task (DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. We compare the results across different models and decoding strategies. Our findings indicate that: (1) When using the greedy search strategy, GPT-4 outperforms 96% of humans, while GPT-3.5-turbo exceeds the average human level. (2) Stochastic sampling and temperature scaling are effective to obtain higher DAT scores for models except GPT-4, but face a trade-off between creativity and stability. These results imply that advanced large language models have divergent semantic associations, which is a fundamental process underlying creativity.
%R 10.18653/v1/2023.findings-emnlp.858
%U https://aclanthology.org/2023.findings-emnlp.858
%U https://doi.org/10.18653/v1/2023.findings-emnlp.858
%P 12881-12888

Markdown (Informal)

[Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association?](https://aclanthology.org/2023.findings-emnlp.858) (Chen & Ding, Findings 2023)

• Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association? (Chen & Ding, Findings 2023)

ACL

• Honghua Chen and Nai Ding. 2023. Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association?. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12881–12888, Singapore. Association for Computational Linguistics.