<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="toral-2020-reassessing">
    <titleInfo>
        <title>Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Antonio</namePart>
        <namePart type="family">Toral</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 22nd Annual Conference of the European Association for Machine Translation</title>
        </titleInfo>
        <originInfo>
            <publisher>European Association for Machine Translation</publisher>
            <place>
                <placeTerm type="text">Lisboa, Portugal</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We reassess the claims of human parity and super-human performance made at the news shared task of WMT2019 for three translation directions: English→German, English→Russian and German→English. First we identify three potential issues in the human evaluation of that shared task: (i) the limited amount of intersen- tential context available, (ii) the limited translation proficiency of the evaluators and (iii) the use of a reference transla- tion. We then conduct a modified eval- uation taking these issues into account. Our results indicate that all the claims of human parity and super-human perfor- mance made at WMT2019 should be re- futed, except the claim of human parity for English→German. Based on our findings, we put forward a set of recommendations and open questions for future assessments of human parity in machine translation.</abstract>
    <identifier type="citekey">toral-2020-reassessing</identifier>
    <location>
        <url>https://aclanthology.org/2020.eamt-1.20</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>185</start>
            <end>194</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019
%A Toral, Antonio
%S Proceedings of the 22nd Annual Conference of the European Association for Machine Translation
%D 2020
%8 November
%I European Association for Machine Translation
%C Lisboa, Portugal
%F toral-2020-reassessing
%X We reassess the claims of human parity and super-human performance made at the news shared task of WMT2019 for three translation directions: English→German, English→Russian and German→English. First we identify three potential issues in the human evaluation of that shared task: (i) the limited amount of intersen- tential context available, (ii) the limited translation proficiency of the evaluators and (iii) the use of a reference transla- tion. We then conduct a modified eval- uation taking these issues into account. Our results indicate that all the claims of human parity and super-human perfor- mance made at WMT2019 should be re- futed, except the claim of human parity for English→German. Based on our findings, we put forward a set of recommendations and open questions for future assessments of human parity in machine translation.
%U https://aclanthology.org/2020.eamt-1.20
%P 185-194

Markdown (Informal)

[Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019](https://aclanthology.org/2020.eamt-1.20) (Toral, EAMT 2020)

• Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019 (Toral, EAMT 2020)

ACL

• Antonio Toral. 2020. Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 185–194, Lisboa, Portugal. European Association for Machine Translation.