<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="thapliyal-etal-2022-crossmodal">
    <titleInfo>
        <title>Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ashish</namePart>
        <namePart type="given">V</namePart>
        <namePart type="family">Thapliyal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jordi</namePart>
        <namePart type="family">Pont Tuset</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xi</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Radu</namePart>
        <namePart type="family">Soricut</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Research in massively multilingual image captioning has been severely hampered by a lack of high-quality evaluation datasets. In this paper we present the Crossmodal-3600 dataset (XM3600 in short), a geographically diverse set of 3600 images annotated with human-generated reference captions in 36 languages. The images were selected from across the world, covering regions where the 36 languages are spoken, and annotated with captions that achieve consistency in terms of style across all languages, while avoiding annotation artifacts due to direct translation. We apply this benchmark to model selection for massively multilingual image captioning models, and show superior correlation results with human evaluations when using XM3600 as golden references for automatic metrics.</abstract>
    <identifier type="citekey">thapliyal-etal-2022-crossmodal</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.45</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.45</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>715</start>
            <end>729</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset
%A Thapliyal, Ashish V.
%A Pont Tuset, Jordi
%A Chen, Xi
%A Soricut, Radu
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F thapliyal-etal-2022-crossmodal
%X Research in massively multilingual image captioning has been severely hampered by a lack of high-quality evaluation datasets. In this paper we present the Crossmodal-3600 dataset (XM3600 in short), a geographically diverse set of 3600 images annotated with human-generated reference captions in 36 languages. The images were selected from across the world, covering regions where the 36 languages are spoken, and annotated with captions that achieve consistency in terms of style across all languages, while avoiding annotation artifacts due to direct translation. We apply this benchmark to model selection for massively multilingual image captioning models, and show superior correlation results with human evaluations when using XM3600 as golden references for automatic metrics.
%R 10.18653/v1/2022.emnlp-main.45
%U https://aclanthology.org/2022.emnlp-main.45
%U https://doi.org/10.18653/v1/2022.emnlp-main.45
%P 715-729

Markdown (Informal)

[Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset](https://aclanthology.org/2022.emnlp-main.45) (Thapliyal et al., EMNLP 2022)

• Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset (Thapliyal et al., EMNLP 2022)

ACL

• Ashish V. Thapliyal, Jordi Pont Tuset, Xi Chen, and Radu Soricut. 2022. Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 715–729, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.