<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="macken-2024-machine">
    <titleInfo>
        <title>Machine Translation Meets Large Language Models: Evaluating ChatGPT’s Ability to Automatically Post-Edit Literary Texts</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lieve</namePart>
        <namePart type="family">Macken</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop on Creative-text Translation and Technology</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Bram</namePart>
            <namePart type="family">Vanroy</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marie-Aude</namePart>
            <namePart type="family">Lefer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lieve</namePart>
            <namePart type="family">Macken</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paola</namePart>
            <namePart type="family">Ruffo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>European Association for Machine Translation</publisher>
            <place>
                <placeTerm type="text">Sheffield, United Kingdom</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large language models such as GPT-4 have been trained on vast corpora, giving them excellent language understanding. This study explores the use of ChatGPT for post-editing machine translations of literary texts. Three short stories, machine translated from English into Dutch, were post-edited by 7-8 professional translators and ChatGPT. Automatic metrics were used to evaluate the number and type of edits made, and semantic and syntactic similarity between the machine translation and the corresponding post-edited versions. A manual analysis classified errors in the machine translation and changes made by the post-editors. The results show that ChatGPT made more changes than the average post-editor. ChatGPT improved lexical richness over machine translation for all texts. The analysis of editing types showed that ChatGPT replaced more words with synonyms, corrected fewer machine errors and introduced more problems than professionals.</abstract>
    <identifier type="citekey">macken-2024-machine</identifier>
    <location>
        <url>https://aclanthology.org/2024.ctt-1.7</url>
    </location>
    <part>
        <date>2024-06</date>
        <extent unit="page">
            <start>65</start>
            <end>81</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Machine Translation Meets Large Language Models: Evaluating ChatGPT’s Ability to Automatically Post-Edit Literary Texts
%A Macken, Lieve
%Y Vanroy, Bram
%Y Lefer, Marie-Aude
%Y Macken, Lieve
%Y Ruffo, Paola
%S Proceedings of the 1st Workshop on Creative-text Translation and Technology
%D 2024
%8 June
%I European Association for Machine Translation
%C Sheffield, United Kingdom
%F macken-2024-machine
%X Large language models such as GPT-4 have been trained on vast corpora, giving them excellent language understanding. This study explores the use of ChatGPT for post-editing machine translations of literary texts. Three short stories, machine translated from English into Dutch, were post-edited by 7-8 professional translators and ChatGPT. Automatic metrics were used to evaluate the number and type of edits made, and semantic and syntactic similarity between the machine translation and the corresponding post-edited versions. A manual analysis classified errors in the machine translation and changes made by the post-editors. The results show that ChatGPT made more changes than the average post-editor. ChatGPT improved lexical richness over machine translation for all texts. The analysis of editing types showed that ChatGPT replaced more words with synonyms, corrected fewer machine errors and introduced more problems than professionals.
%U https://aclanthology.org/2024.ctt-1.7
%P 65-81

Markdown (Informal)

[Machine Translation Meets Large Language Models: Evaluating ChatGPT’s Ability to Automatically Post-Edit Literary Texts](https://aclanthology.org/2024.ctt-1.7) (Macken, CTT-WS 2024)

• Machine Translation Meets Large Language Models: Evaluating ChatGPT’s Ability to Automatically Post-Edit Literary Texts (Macken, CTT-WS 2024)

ACL

• Lieve Macken. 2024. Machine Translation Meets Large Language Models: Evaluating ChatGPT’s Ability to Automatically Post-Edit Literary Texts. In Proceedings of the 1st Workshop on Creative-text Translation and Technology, pages 65–81, Sheffield, United Kingdom. European Association for Machine Translation.