<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mutal-etal-2019-differences">
    <titleInfo>
        <title>Differences between SMT and NMT Output - a Translators’ Point of View</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="family">Mutal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lise</namePart>
        <namePart type="family">Volkart</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pierrette</namePart>
        <namePart type="family">Bouillon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sabrina</namePart>
        <namePart type="family">Girletti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paula</namePart>
        <namePart type="family">Estrella</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Human-Informed Translation and Interpreting Technology Workshop (HiT-IT 2019)</title>
        </titleInfo>
        <originInfo>
            <publisher>Incoma Ltd., Shoumen, Bulgaria</publisher>
            <place>
                <placeTerm type="text">Varna, Bulgaria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this study, we compare the output quality of two MT systems, a statistical (SMT) and a neural (NMT) engine, customised for Swiss Post’s Language Service using the same training data. We focus on the point of view of professional translators and investigate how they perceive the differences between the MT output and a human reference (namely deletions, substitutions, insertions and word order). Our findings show that translators more frequently consider these differences to be errors in SMT than NMT, and that deletions are the most serious errors in both architectures. We also observe lower agreement on differences to be corrected in NMT than in SMT, suggesting that errors are easier to identify in SMT. These findings confirm the ability of NMT to produce correct paraphrases, which could also explain why BLEU is often considered as an inadequate metric to evaluate the performance of NMT systems.</abstract>
    <identifier type="citekey">mutal-etal-2019-differences</identifier>
    <identifier type="doi">10.26615/issn.2683-0078.2019_009</identifier>
    <location>
        <url>https://aclanthology.org/W19-8709</url>
    </location>
    <part>
        <date>2019-09</date>
        <extent unit="page">
            <start>75</start>
            <end>81</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Differences between SMT and NMT Output - a Translators’ Point of View
%A Mutal, Jonathan
%A Volkart, Lise
%A Bouillon, Pierrette
%A Girletti, Sabrina
%A Estrella, Paula
%S Proceedings of the Human-Informed Translation and Interpreting Technology Workshop (HiT-IT 2019)
%D 2019
%8 September
%I Incoma Ltd., Shoumen, Bulgaria
%C Varna, Bulgaria
%F mutal-etal-2019-differences
%X In this study, we compare the output quality of two MT systems, a statistical (SMT) and a neural (NMT) engine, customised for Swiss Post’s Language Service using the same training data. We focus on the point of view of professional translators and investigate how they perceive the differences between the MT output and a human reference (namely deletions, substitutions, insertions and word order). Our findings show that translators more frequently consider these differences to be errors in SMT than NMT, and that deletions are the most serious errors in both architectures. We also observe lower agreement on differences to be corrected in NMT than in SMT, suggesting that errors are easier to identify in SMT. These findings confirm the ability of NMT to produce correct paraphrases, which could also explain why BLEU is often considered as an inadequate metric to evaluate the performance of NMT systems.
%R 10.26615/issn.2683-0078.2019_009
%U https://aclanthology.org/W19-8709
%U https://doi.org/10.26615/issn.2683-0078.2019_009
%P 75-81

Markdown (Informal)

[Differences between SMT and NMT Output - a Translators’ Point of View](https://aclanthology.org/W19-8709) (Mutal et al., RANLP 2019)

• Differences between SMT and NMT Output - a Translators’ Point of View (Mutal et al., RANLP 2019)

ACL

• Jonathan Mutal, Lise Volkart, Pierrette Bouillon, Sabrina Girletti, and Paula Estrella. 2019. Differences between SMT and NMT Output - a Translators’ Point of View. In Proceedings of the Human-Informed Translation and Interpreting Technology Workshop (HiT-IT 2019), pages 75–81, Varna, Bulgaria. Incoma Ltd., Shoumen, Bulgaria.