<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="xu-etal-2023-understanding">
    <titleInfo>
        <title>Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Weijia</namePart>
        <namePart type="family">Xu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sweta</namePart>
        <namePart type="family">Agrawal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Eleftheria</namePart>
        <namePart type="family">Briakou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marianna</namePart>
        <namePart type="given">J</namePart>
        <namePart type="family">Martindale</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marine</namePart>
        <namePart type="family">Carpuat</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Neural sequence generation models are known to “hallucinate”, by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. In this work, we first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinated vs. non-hallucinated outputs generated via source perturbations. We then show that these symptoms are reliable indicators of natural hallucinations, by using them to design a lightweight hallucination detector which outperforms both model-free baselines and strong classifiers based on quality estimation or large pre-trained models on manually annotated English-Chinese and German-English translation test beds.</abstract>
    <identifier type="citekey">xu-etal-2023-understanding</identifier>
    <identifier type="doi">10.1162/tacl_a_00563</identifier>
    <location>
        <url>https://aclanthology.org/2023.tacl-1.32</url>
    </location>
    <part>
        <date>2023</date>
        <detail type="volume"><number>11</number></detail>
        <extent unit="page">
            <start>546</start>
            <end>564</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection
%A Xu, Weijia
%A Agrawal, Sweta
%A Briakou, Eleftheria
%A Martindale, Marianna J.
%A Carpuat, Marine
%J Transactions of the Association for Computational Linguistics
%D 2023
%V 11
%I MIT Press
%C Cambridge, MA
%F xu-etal-2023-understanding
%X Neural sequence generation models are known to “hallucinate”, by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. In this work, we first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinated vs. non-hallucinated outputs generated via source perturbations. We then show that these symptoms are reliable indicators of natural hallucinations, by using them to design a lightweight hallucination detector which outperforms both model-free baselines and strong classifiers based on quality estimation or large pre-trained models on manually annotated English-Chinese and German-English translation test beds.
%R 10.1162/tacl_a_00563
%U https://aclanthology.org/2023.tacl-1.32
%U https://doi.org/10.1162/tacl_a_00563
%P 546-564

Markdown (Informal)

[Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://aclanthology.org/2023.tacl-1.32) (Xu et al., TACL 2023)

• Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection (Xu et al., TACL 2023)

ACL

• Weijia Xu, Sweta Agrawal, Eleftheria Briakou, Marianna J. Martindale, and Marine Carpuat. 2023. Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection. Transactions of the Association for Computational Linguistics, 11:546–564.