<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pistotti-etal-2025-benefits">
    <titleInfo>
        <title>The Benefits of Being Uncertain: Perplexity as a Signal for Naturalness in Multilingual Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Timothy</namePart>
        <namePart type="family">Pistotti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Michael</namePart>
        <namePart type="family">J. Witbrock</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dr</namePart>
        <namePart type="family">Padriac Amato Tahua O’Leary</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jason</namePart>
        <namePart type="family">Brown</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Noidea</namePart>
            <namePart type="family">Noidea</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Suzhou, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-349-4</identifier>
    </relatedItem>
    <abstract>Model-internal uncertainty metrics like perplexity potentially offer low-cost signals for Machine Translation Quality Estimation (TQE). This paper analyses perplexity in the No Language Left Behind (NLLB) multilingual model. We quantify a significant model-human perplexity gap, where the model is consistently more confident in its own, often literal, machine-generated translation than in diverse, high-quality human versions. We then demonstrate that the utility of perplexity as a TQE signal is highly context-dependent, being strongest for low-resource pairs. Finally, we present an illustrative case study where a flawed translation is refined by providing potentially useful information in a targeted prompt, simulating a knowledge-based repair. We show that as the translation’s quality and naturalness improve (a +0.15 COMET score increase), its perplexity also increases, challenging the simple assumption that lower perplexity indicates higher quality and motivating a more nuanced view of uncertainty as signalling a text’s departure from rigid translationese.</abstract>
    <identifier type="citekey">pistotti-etal-2025-benefits</identifier>
    <location>
        <url>https://aclanthology.org/2025.uncertainlp-main.7/</url>
    </location>
    <part>
        <date>2025-11</date>
        <extent unit="page">
            <start>61</start>
            <end>65</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Benefits of Being Uncertain: Perplexity as a Signal for Naturalness in Multilingual Machine Translation
%A Pistotti, Timothy
%A J. Witbrock, Michael
%A Padriac Amato Tahua O’Leary, Dr
%A Brown, Jason
%Y Noidea, Noidea
%S Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025)
%D 2025
%8 November
%I Association for Computational Linguistics
%C Suzhou, China
%@ 979-8-89176-349-4
%F pistotti-etal-2025-benefits
%X Model-internal uncertainty metrics like perplexity potentially offer low-cost signals for Machine Translation Quality Estimation (TQE). This paper analyses perplexity in the No Language Left Behind (NLLB) multilingual model. We quantify a significant model-human perplexity gap, where the model is consistently more confident in its own, often literal, machine-generated translation than in diverse, high-quality human versions. We then demonstrate that the utility of perplexity as a TQE signal is highly context-dependent, being strongest for low-resource pairs. Finally, we present an illustrative case study where a flawed translation is refined by providing potentially useful information in a targeted prompt, simulating a knowledge-based repair. We show that as the translation’s quality and naturalness improve (a +0.15 COMET score increase), its perplexity also increases, challenging the simple assumption that lower perplexity indicates higher quality and motivating a more nuanced view of uncertainty as signalling a text’s departure from rigid translationese.
%U https://aclanthology.org/2025.uncertainlp-main.7/
%P 61-65

Markdown (Informal)

[The Benefits of Being Uncertain: Perplexity as a Signal for Naturalness in Multilingual Machine Translation](https://aclanthology.org/2025.uncertainlp-main.7/) (Pistotti et al., UncertaiNLP 2025)

• The Benefits of Being Uncertain: Perplexity as a Signal for Naturalness in Multilingual Machine Translation (Pistotti et al., UncertaiNLP 2025)

ACL

• Timothy Pistotti, Michael J. Witbrock, Dr Padriac Amato Tahua O’Leary, and Jason Brown. 2025. The Benefits of Being Uncertain: Perplexity as a Signal for Naturalness in Multilingual Machine Translation. In Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025), pages 61–65, Suzhou, China. Association for Computational Linguistics.