<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nakayama-etal-2020-visually">
    <titleInfo>
        <title>A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hideki</namePart>
        <namePart type="family">Nakayama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Akihiro</namePart>
        <namePart type="family">Tamura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Takashi</namePart>
        <namePart type="family">Ninomiya</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <language>
        <languageTerm type="text">English</languageTerm>
        <languageTerm type="code" authority="iso639-2b">eng</languageTerm>
    </language>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Twelfth Language Resources and Evaluation Conference</title>
        </titleInfo>
        <originInfo>
            <publisher>European Language Resources Association</publisher>
            <place>
                <placeTerm type="text">Marseille, France</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-10-95546-34-4</identifier>
    </relatedItem>
    <abstract>Visually-grounded natural language processing has become an important research direction in the past few years. However, majorities of the available cross-modal resources (e.g., image-caption datasets) are built in English and cannot be directly utilized in multilingual or non-English scenarios. In this study, we present a novel multilingual multimodal corpus by extending the Flickr30k Entities image-caption dataset with Japanese translations, which we name Flickr30k Entities JP (F30kEnt-JP). To the best of our knowledge, this is the first multilingual image-caption dataset where the captions in the two languages are parallel and have the shared annotations of many-to-many phrase-to-region linking. We believe that phrase-to-region as well as phrase-to-phrase supervision can play a vital role in fine-grained grounding of language and vision, and will promote many tasks such as multilingual image captioning and multimodal machine translation. To verify our dataset, we performed phrase localization experiments in both languages and investigated the effectiveness of our Japanese annotations as well as multilingual learning realized by our dataset.</abstract>
    <identifier type="citekey">nakayama-etal-2020-visually</identifier>
    <location>
        <url>https://aclanthology.org/2020.lrec-1.518</url>
    </location>
    <part>
        <date>2020-05</date>
        <extent unit="page">
            <start>4204</start>
            <end>4210</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking
%A Nakayama, Hideki
%A Tamura, Akihiro
%A Ninomiya, Takashi
%S Proceedings of the Twelfth Language Resources and Evaluation Conference
%D 2020
%8 May
%I European Language Resources Association
%C Marseille, France
%@ 979-10-95546-34-4
%G English
%F nakayama-etal-2020-visually
%X Visually-grounded natural language processing has become an important research direction in the past few years. However, majorities of the available cross-modal resources (e.g., image-caption datasets) are built in English and cannot be directly utilized in multilingual or non-English scenarios. In this study, we present a novel multilingual multimodal corpus by extending the Flickr30k Entities image-caption dataset with Japanese translations, which we name Flickr30k Entities JP (F30kEnt-JP). To the best of our knowledge, this is the first multilingual image-caption dataset where the captions in the two languages are parallel and have the shared annotations of many-to-many phrase-to-region linking. We believe that phrase-to-region as well as phrase-to-phrase supervision can play a vital role in fine-grained grounding of language and vision, and will promote many tasks such as multilingual image captioning and multimodal machine translation. To verify our dataset, we performed phrase localization experiments in both languages and investigated the effectiveness of our Japanese annotations as well as multilingual learning realized by our dataset.
%U https://aclanthology.org/2020.lrec-1.518
%P 4204-4210

Markdown (Informal)

[A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking](https://aclanthology.org/2020.lrec-1.518) (Nakayama et al., LREC 2020)

• A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking (Nakayama et al., LREC 2020)

ACL

• Hideki Nakayama, Akihiro Tamura, and Takashi Ninomiya. 2020. A Visually-Grounded Parallel Corpus with Phrase-to-Region Linking. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4204–4210, Marseille, France. European Language Resources Association.