<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ramos-etal-2023-lmcap">
    <titleInfo>
        <title>LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Rita</namePart>
        <namePart type="family">Ramos</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Bruno</namePart>
        <namePart type="family">Martins</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Desmond</namePart>
        <namePart type="family">Elliott</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Multilingual image captioning has recently been tackled by training with large-scale machine translated data, which is an expensive, noisy, and time-consuming process. Without requiring any multilingual caption data, we propose LMCap, an image-blind few-shot multilingual captioning model that works by prompting a language model with retrieved captions. Specifically, instead of following the standard encoder-decoder paradigm, given an image, LMCap first retrieves the captions of similar images using a multilingual CLIP encoder. These captions are then combined into a prompt for an XGLM decoder, in order to generate captions in the desired language. In other words, the generation model does not directly process the image, instead it processes retrieved captions. Experiments on the XM3600 dataset of geographically diverse images show that our model is competitive with fully-supervised multilingual captioning models, without requiring any supervised training on any captioning data.</abstract>
    <identifier type="citekey">ramos-etal-2023-lmcap</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.104</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.104</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>1635</start>
            <end>1651</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting
%A Ramos, Rita
%A Martins, Bruno
%A Elliott, Desmond
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F ramos-etal-2023-lmcap
%X Multilingual image captioning has recently been tackled by training with large-scale machine translated data, which is an expensive, noisy, and time-consuming process. Without requiring any multilingual caption data, we propose LMCap, an image-blind few-shot multilingual captioning model that works by prompting a language model with retrieved captions. Specifically, instead of following the standard encoder-decoder paradigm, given an image, LMCap first retrieves the captions of similar images using a multilingual CLIP encoder. These captions are then combined into a prompt for an XGLM decoder, in order to generate captions in the desired language. In other words, the generation model does not directly process the image, instead it processes retrieved captions. Experiments on the XM3600 dataset of geographically diverse images show that our model is competitive with fully-supervised multilingual captioning models, without requiring any supervised training on any captioning data.
%R 10.18653/v1/2023.findings-acl.104
%U https://aclanthology.org/2023.findings-acl.104
%U https://doi.org/10.18653/v1/2023.findings-acl.104
%P 1635-1651

Markdown (Informal)

[LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting](https://aclanthology.org/2023.findings-acl.104) (Ramos et al., Findings 2023)

• LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting (Ramos et al., Findings 2023)

ACL

• Rita Ramos, Bruno Martins, and Desmond Elliott. 2023. LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1635–1651, Toronto, Canada. Association for Computational Linguistics.