<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sarti-etal-2025-qe4pe">
    <titleInfo>
        <title>QE4PE: Word-level Quality Estimation for Human Post-Editing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gabriele</namePart>
        <namePart type="family">Sarti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vilém</namePart>
        <namePart type="family">Zouhar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Grzegorz</namePart>
        <namePart type="family">Chrupała</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ana</namePart>
        <namePart type="family">Guerberof-Arenas</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Malvina</namePart>
        <namePart type="family">Nissim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Arianna</namePart>
        <namePart type="family">Bisazza</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Word-level quality estimation (QE) methods aim to detect erroneous spans in machine translations, which can direct and facilitate human post-editing. While the accuracy of word-level QE systems has been assessed extensively, their usability and downstream influence on the speed, quality, and editing choices of human post-editing remain understudied. In this study, we investigate the impact of word-level QE on machine translation (MT) post-editing in a realistic setting involving 42 professional post-editors across two translation directions. We compare four error-span highlight modalities, including supervised and uncertainty-based word-level QE methods, for identifying potential errors in the outputs of a state-of-the-art neural MT model. Post-editing effort and productivity are estimated from behavioral logs, while quality improvements are assessed by word- and segment-level human annotation. We find that domain, language and editors’ speed are critical factors in determining highlights’ effectiveness, with modest differences between human-made and automated QE highlights underlining a gap between accuracy and usability in professional workflows.</abstract>
    <identifier type="citekey">sarti-etal-2025-qe4pe</identifier>
    <identifier type="doi">10.1162/tacl.a.46</identifier>
    <location>
        <url>https://aclanthology.org/2025.tacl-1.64/</url>
    </location>
    <part>
        <date>2025</date>
        <detail type="volume"><number>13</number></detail>
        <extent unit="page">
            <start>1410</start>
            <end>1435</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T QE4PE: Word-level Quality Estimation for Human Post-Editing
%A Sarti, Gabriele
%A Zouhar, Vilém
%A Chrupała, Grzegorz
%A Guerberof-Arenas, Ana
%A Nissim, Malvina
%A Bisazza, Arianna
%J Transactions of the Association for Computational Linguistics
%D 2025
%V 13
%I MIT Press
%C Cambridge, MA
%F sarti-etal-2025-qe4pe
%X Word-level quality estimation (QE) methods aim to detect erroneous spans in machine translations, which can direct and facilitate human post-editing. While the accuracy of word-level QE systems has been assessed extensively, their usability and downstream influence on the speed, quality, and editing choices of human post-editing remain understudied. In this study, we investigate the impact of word-level QE on machine translation (MT) post-editing in a realistic setting involving 42 professional post-editors across two translation directions. We compare four error-span highlight modalities, including supervised and uncertainty-based word-level QE methods, for identifying potential errors in the outputs of a state-of-the-art neural MT model. Post-editing effort and productivity are estimated from behavioral logs, while quality improvements are assessed by word- and segment-level human annotation. We find that domain, language and editors’ speed are critical factors in determining highlights’ effectiveness, with modest differences between human-made and automated QE highlights underlining a gap between accuracy and usability in professional workflows.
%R 10.1162/tacl.a.46
%U https://aclanthology.org/2025.tacl-1.64/
%U https://doi.org/10.1162/tacl.a.46
%P 1410-1435

Markdown (Informal)

[QE4PE: Word-level Quality Estimation for Human Post-Editing](https://aclanthology.org/2025.tacl-1.64/) (Sarti et al., TACL 2025)

• QE4PE: Word-level Quality Estimation for Human Post-Editing (Sarti et al., TACL 2025)

ACL

• Gabriele Sarti, Vilém Zouhar, Grzegorz Chrupała, Ana Guerberof-Arenas, Malvina Nissim, and Arianna Bisazza. 2025. QE4PE: Word-level Quality Estimation for Human Post-Editing. Transactions of the Association for Computational Linguistics, 13:1410–1435.