<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pu-demberg-2023-chatgpt">
    <titleInfo>
        <title>ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Dongqi</namePart>
        <namePart type="family">Pu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vera</namePart>
        <namePart type="family">Demberg</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Vishakh</namePart>
            <namePart type="family">Padmakumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gisela</namePart>
            <namePart type="family">Vallejo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yao</namePart>
            <namePart type="family">Fu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large-scale language models, like ChatGPT, have garnered significant media attention and stunned the public with their remarkable capacity for generating coherent text from short natural language prompts. In this paper, we aim to conduct a systematic inspection of ChatGPT’s performance in two controllable generation tasks, with respect to ChatGPT’s ability to adapt its output to different target audiences (expert vs. layman) and writing styles (formal vs. informal). Additionally, we evaluate the faithfulness of the generated text, and compare the model’s performance with human-authored texts. Our findings indicate that the stylistic variations produced by humans are considerably larger than those demonstrated by ChatGPT, and the generated texts diverge from human samples in several characteristics, such as the distribution of word types. Moreover, we observe that ChatGPT sometimes incorporates factual errors or hallucinations when adapting the text to suit a specific style.</abstract>
    <identifier type="citekey">pu-demberg-2023-chatgpt</identifier>
    <identifier type="doi">10.18653/v1/2023.acl-srw.1</identifier>
    <location>
        <url>https://aclanthology.org/2023.acl-srw.1</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>1</start>
            <end>18</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer
%A Pu, Dongqi
%A Demberg, Vera
%Y Padmakumar, Vishakh
%Y Vallejo, Gisela
%Y Fu, Yao
%S Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F pu-demberg-2023-chatgpt
%X Large-scale language models, like ChatGPT, have garnered significant media attention and stunned the public with their remarkable capacity for generating coherent text from short natural language prompts. In this paper, we aim to conduct a systematic inspection of ChatGPT’s performance in two controllable generation tasks, with respect to ChatGPT’s ability to adapt its output to different target audiences (expert vs. layman) and writing styles (formal vs. informal). Additionally, we evaluate the faithfulness of the generated text, and compare the model’s performance with human-authored texts. Our findings indicate that the stylistic variations produced by humans are considerably larger than those demonstrated by ChatGPT, and the generated texts diverge from human samples in several characteristics, such as the distribution of word types. Moreover, we observe that ChatGPT sometimes incorporates factual errors or hallucinations when adapting the text to suit a specific style.
%R 10.18653/v1/2023.acl-srw.1
%U https://aclanthology.org/2023.acl-srw.1
%U https://doi.org/10.18653/v1/2023.acl-srw.1
%P 1-18

Markdown (Informal)

[ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer](https://aclanthology.org/2023.acl-srw.1) (Pu & Demberg, ACL 2023)

• ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer (Pu & Demberg, ACL 2023)

ACL

• Dongqi Pu and Vera Demberg. 2023. ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 1–18, Toronto, Canada. Association for Computational Linguistics.