<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zouhar-etal-2020-wmt20">
    <titleInfo>
        <title>WMT20 Document-Level Markable Error Exploration</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Vilém</namePart>
        <namePart type="family">Zouhar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tereza</namePart>
        <namePart type="family">Vojtěchová</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ondřej</namePart>
        <namePart type="family">Bojar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fifth Conference on Machine Translation</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Even though sentence-centric metrics are used widely in machine translation evaluation, document-level performance is at least equally important for professional usage. In this paper, we bring attention to detailed document-level evaluation focused on markables (expressions bearing most of the document meaning) and the negative impact of various markable error phenomena on the translation. For an annotation experiment of two phases, we chose Czech and English documents translated by systems submitted to WMT20 News Translation Task. These documents are from the News, Audit and Lease domains. We show that the quality and also the kind of errors varies significantly among the domains. This systematic variance is in contrast to the automatic evaluation results. We inspect which specific markables are problematic for MT systems and conclude with an analysis of the effect of markable error types on the MT performance measured by humans and automatic evaluation tools.</abstract>
    <identifier type="citekey">zouhar-etal-2020-wmt20</identifier>
    <location>
        <url>https://aclanthology.org/2020.wmt-1.41</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>371</start>
            <end>380</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T WMT20 Document-Level Markable Error Exploration
%A Zouhar, Vilém
%A Vojtěchová, Tereza
%A Bojar, Ondřej
%S Proceedings of the Fifth Conference on Machine Translation
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F zouhar-etal-2020-wmt20
%X Even though sentence-centric metrics are used widely in machine translation evaluation, document-level performance is at least equally important for professional usage. In this paper, we bring attention to detailed document-level evaluation focused on markables (expressions bearing most of the document meaning) and the negative impact of various markable error phenomena on the translation. For an annotation experiment of two phases, we chose Czech and English documents translated by systems submitted to WMT20 News Translation Task. These documents are from the News, Audit and Lease domains. We show that the quality and also the kind of errors varies significantly among the domains. This systematic variance is in contrast to the automatic evaluation results. We inspect which specific markables are problematic for MT systems and conclude with an analysis of the effect of markable error types on the MT performance measured by humans and automatic evaluation tools.
%U https://aclanthology.org/2020.wmt-1.41
%P 371-380

Markdown (Informal)

[WMT20 Document-Level Markable Error Exploration](https://aclanthology.org/2020.wmt-1.41) (Zouhar et al., WMT 2020)

• WMT20 Document-Level Markable Error Exploration (Zouhar et al., WMT 2020)

ACL

• Vilém Zouhar, Tereza Vojtěchová, and Ondřej Bojar. 2020. WMT20 Document-Level Markable Error Exploration. In Proceedings of the Fifth Conference on Machine Translation, pages 371–380, Online. Association for Computational Linguistics.