<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yang-etal-2023-human">
    <titleInfo>
        <title>Human-in-the-loop Machine Translation with Large Language Model</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xinyi</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Runzhe</namePart>
        <namePart type="family">Zhan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Derek</namePart>
        <namePart type="given">F</namePart>
        <namePart type="family">Wong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Junchao</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lidia</namePart>
        <namePart type="given">S</namePart>
        <namePart type="family">Chao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of Machine Translation Summit XIX, Vol. 2: Users Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Masaru</namePart>
            <namePart type="family">Yamada</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Felix</namePart>
            <namePart type="family">do Carmo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Asia-Pacific Association for Machine Translation</publisher>
            <place>
                <placeTerm type="text">Macau SAR, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The large language model (LLM) has garnered significant attention due to its in-context learning mechanisms and emergent capabilities. The research community has conducted several pilot studies to apply LLMs to machine translation tasks and evaluate their performance from diverse perspectives. However, previous research has primarily focused on the LLM itself and has not explored human intervention in the inference process of LLM. The characteristics of LLM, such as in-context learning and prompt engineering, closely mirror human cognitive abilities in language tasks, offering an intuitive solution for human-in-the-loop generation. In this study, we propose a human-in-the-loop pipeline that guides LLMs to produce customized outputs with revision instructions. The pipeline initiates by prompting the LLM to produce a draft translation, followed by the utilization of automatic retrieval or human feedback as supervision signals to enhance the LLM’s translation through in-context learning. The human-machine interactions generated in this pipeline are also stored in an external database to expand the in-context retrieval database, enabling us to leverage human supervision in an offline setting. We evaluate the proposed pipeline using the GPT-3.5-turbo API on five domain-specific benchmarks for German-English translation. The results demonstrate the effectiveness of the pipeline in tailoring in-domain translations and improving translation performance compared to direct translation instructions. Additionally, we discuss the experimental results from the following perspectives: 1) the effectiveness of different in-context retrieval methods; 2) the construction of a retrieval database under low-resource scenarios; 3) the observed differences across selected domains; 4) the quantitative analysis of sentence-level and word-level statistics; and 5) the qualitative analysis of representative translation cases.</abstract>
    <identifier type="citekey">yang-etal-2023-human</identifier>
    <location>
        <url>https://aclanthology.org/2023.mtsummit-users.8</url>
    </location>
    <part>
        <date>2023-09</date>
        <extent unit="page">
            <start>88</start>
            <end>98</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Human-in-the-loop Machine Translation with Large Language Model
%A Yang, Xinyi
%A Zhan, Runzhe
%A Wong, Derek F.
%A Wu, Junchao
%A Chao, Lidia S.
%Y Yamada, Masaru
%Y do Carmo, Felix
%S Proceedings of Machine Translation Summit XIX, Vol. 2: Users Track
%D 2023
%8 September
%I Asia-Pacific Association for Machine Translation
%C Macau SAR, China
%F yang-etal-2023-human
%X The large language model (LLM) has garnered significant attention due to its in-context learning mechanisms and emergent capabilities. The research community has conducted several pilot studies to apply LLMs to machine translation tasks and evaluate their performance from diverse perspectives. However, previous research has primarily focused on the LLM itself and has not explored human intervention in the inference process of LLM. The characteristics of LLM, such as in-context learning and prompt engineering, closely mirror human cognitive abilities in language tasks, offering an intuitive solution for human-in-the-loop generation. In this study, we propose a human-in-the-loop pipeline that guides LLMs to produce customized outputs with revision instructions. The pipeline initiates by prompting the LLM to produce a draft translation, followed by the utilization of automatic retrieval or human feedback as supervision signals to enhance the LLM’s translation through in-context learning. The human-machine interactions generated in this pipeline are also stored in an external database to expand the in-context retrieval database, enabling us to leverage human supervision in an offline setting. We evaluate the proposed pipeline using the GPT-3.5-turbo API on five domain-specific benchmarks for German-English translation. The results demonstrate the effectiveness of the pipeline in tailoring in-domain translations and improving translation performance compared to direct translation instructions. Additionally, we discuss the experimental results from the following perspectives: 1) the effectiveness of different in-context retrieval methods; 2) the construction of a retrieval database under low-resource scenarios; 3) the observed differences across selected domains; 4) the quantitative analysis of sentence-level and word-level statistics; and 5) the qualitative analysis of representative translation cases.
%U https://aclanthology.org/2023.mtsummit-users.8
%P 88-98

Markdown (Informal)

[Human-in-the-loop Machine Translation with Large Language Model](https://aclanthology.org/2023.mtsummit-users.8) (Yang et al., MTSummit 2023)

• Human-in-the-loop Machine Translation with Large Language Model (Yang et al., MTSummit 2023)

ACL

• Xinyi Yang, Runzhe Zhan, Derek F. Wong, Junchao Wu, and Lidia S. Chao. 2023. Human-in-the-loop Machine Translation with Large Language Model. In Proceedings of Machine Translation Summit XIX, Vol. 2: Users Track, pages 88–98, Macau SAR, China. Asia-Pacific Association for Machine Translation.