<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="elliott-2018-adversarial">
    <titleInfo>
        <title>Adversarial Evaluation of Multimodal Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Desmond</namePart>
        <namePart type="family">Elliott</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-oct-nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ellen</namePart>
            <namePart type="family">Riloff</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Chiang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Julia</namePart>
            <namePart type="family">Hockenmaier</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun’ichi</namePart>
            <namePart type="family">Tsujii</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Brussels, Belgium</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The promise of combining language and vision in multimodal machine translation is that systems will produce better translations by leveraging the image data. However, the evidence surrounding whether the images are useful is unconvincing due to inconsistencies between text-similarity metrics and human judgements. We present an adversarial evaluation to directly examine the utility of the image data in this task. Our evaluation tests whether systems perform better when paired with congruent images or incongruent images. This evaluation shows that only one out of three publicly available systems is sensitive to this perturbation of the data. We recommend that multimodal translation systems should be able to pass this sanity check in the future.</abstract>
    <identifier type="citekey">elliott-2018-adversarial</identifier>
    <identifier type="doi">10.18653/v1/D18-1329</identifier>
    <location>
        <url>https://aclanthology.org/D18-1329/</url>
    </location>
    <part>
        <date>2018-oct-nov</date>
        <extent unit="page">
            <start>2974</start>
            <end>2978</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Adversarial Evaluation of Multimodal Machine Translation
%A Elliott, Desmond
%Y Riloff, Ellen
%Y Chiang, David
%Y Hockenmaier, Julia
%Y Tsujii, Jun’ichi
%S Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
%D 2018
%8 oct nov
%I Association for Computational Linguistics
%C Brussels, Belgium
%F elliott-2018-adversarial
%X The promise of combining language and vision in multimodal machine translation is that systems will produce better translations by leveraging the image data. However, the evidence surrounding whether the images are useful is unconvincing due to inconsistencies between text-similarity metrics and human judgements. We present an adversarial evaluation to directly examine the utility of the image data in this task. Our evaluation tests whether systems perform better when paired with congruent images or incongruent images. This evaluation shows that only one out of three publicly available systems is sensitive to this perturbation of the data. We recommend that multimodal translation systems should be able to pass this sanity check in the future.
%R 10.18653/v1/D18-1329
%U https://aclanthology.org/D18-1329/
%U https://doi.org/10.18653/v1/D18-1329
%P 2974-2978

Markdown (Informal)

[Adversarial Evaluation of Multimodal Machine Translation](https://aclanthology.org/D18-1329/) (Elliott, EMNLP 2018)

• Adversarial Evaluation of Multimodal Machine Translation (Elliott, EMNLP 2018)

ACL

• Desmond Elliott. 2018. Adversarial Evaluation of Multimodal Machine Translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2974–2978, Brussels, Belgium. Association for Computational Linguistics.