<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yuan-etal-2020-bridge">
    <titleInfo>
        <title>Bridge the Gap: High-level Semantic Planning for Image Captioning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Chenxi</namePart>
        <namePart type="family">Yuan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yang</namePart>
        <namePart type="family">Bai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chun</namePart>
        <namePart type="family">Yuan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 28th International Conference on Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <publisher>International Committee on Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Barcelona, Spain (Online)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent image captioning models have made much progress for exploring the multi-modal interaction, such as attention mechanisms. Though these mechanisms can boost the interaction, there are still two gaps between the visual and language domains: (1) the gap between the visual features and textual semantics, (2) the gap between the disordering of visual features and the ordering of texts. To bridge the gaps we propose a high-level semantic planning (HSP) mechanism that incorporates both a semantic reconstruction and an explicit order planning. We integrate the planning mechanism to the attention based caption model and propose the High-level Semantic PLanning based Attention Network (HS-PLAN). First, an attention based reconstruction module is designed to reconstruct the visual features with high-level semantic information. Then we apply a pointer network to serialize the features and obtain the explicit order plan to guide the generation. Experiments conducted on MS COCO show that our model outperforms previous methods and achieves the state-of-the-art performance of 133.4% CIDEr-D score.</abstract>
    <identifier type="citekey">yuan-etal-2020-bridge</identifier>
    <identifier type="doi">10.18653/v1/2020.coling-main.281</identifier>
    <location>
        <url>https://aclanthology.org/2020.coling-main.281</url>
    </location>
    <part>
        <date>2020-12</date>
        <extent unit="page">
            <start>3157</start>
            <end>3167</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Bridge the Gap: High-level Semantic Planning for Image Captioning
%A Yuan, Chenxi
%A Bai, Yang
%A Yuan, Chun
%S Proceedings of the 28th International Conference on Computational Linguistics
%D 2020
%8 December
%I International Committee on Computational Linguistics
%C Barcelona, Spain (Online)
%F yuan-etal-2020-bridge
%X Recent image captioning models have made much progress for exploring the multi-modal interaction, such as attention mechanisms. Though these mechanisms can boost the interaction, there are still two gaps between the visual and language domains: (1) the gap between the visual features and textual semantics, (2) the gap between the disordering of visual features and the ordering of texts. To bridge the gaps we propose a high-level semantic planning (HSP) mechanism that incorporates both a semantic reconstruction and an explicit order planning. We integrate the planning mechanism to the attention based caption model and propose the High-level Semantic PLanning based Attention Network (HS-PLAN). First, an attention based reconstruction module is designed to reconstruct the visual features with high-level semantic information. Then we apply a pointer network to serialize the features and obtain the explicit order plan to guide the generation. Experiments conducted on MS COCO show that our model outperforms previous methods and achieves the state-of-the-art performance of 133.4% CIDEr-D score.
%R 10.18653/v1/2020.coling-main.281
%U https://aclanthology.org/2020.coling-main.281
%U https://doi.org/10.18653/v1/2020.coling-main.281
%P 3157-3167

Markdown (Informal)

[Bridge the Gap: High-level Semantic Planning for Image Captioning](https://aclanthology.org/2020.coling-main.281) (Yuan et al., COLING 2020)

• Bridge the Gap: High-level Semantic Planning for Image Captioning (Yuan et al., COLING 2020)

ACL

• Chenxi Yuan, Yang Bai, and Chun Yuan. 2020. Bridge the Gap: High-level Semantic Planning for Image Captioning. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3157–3167, Barcelona, Spain (Online). International Committee on Computational Linguistics.