<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ilinykh-dobnik-2020-image">
    <titleInfo>
        <title>When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nikolai</namePart>
        <namePart type="family">Ilinykh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Simon</namePart>
        <namePart type="family">Dobnik</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 13th International Conference on Natural Language Generation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Brian</namePart>
            <namePart type="family">Davis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yvette</namePart>
            <namePart type="family">Graham</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">John</namePart>
            <namePart type="family">Kelleher</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yaji</namePart>
            <namePart type="family">Sripada</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Generating multi-sentence image descriptions is a challenging task, which requires a good model to produce coherent and accurate paragraphs, describing salient objects in the image. We argue that multiple sources of information are beneficial when describing visual scenes with long sequences. These include (i) perceptual information and (ii) semantic (language) information about how to describe what is in the image. We also compare the effects of using two different pooling mechanisms on either a single modality or their combination. We demonstrate that the model which utilises both visual and language inputs can be used to generate accurate and diverse paragraphs when combined with a particular pooling mechanism. The results of our automatic and human evaluation show that learning to embed semantic information along with visual stimuli into the paragraph generation model is not trivial, raising a variety of proposals for future experiments.</abstract>
    <identifier type="citekey">ilinykh-dobnik-2020-image</identifier>
    <identifier type="doi">10.18653/v1/2020.inlg-1.40</identifier>
    <location>
        <url>https://aclanthology.org/2020.inlg-1.40</url>
    </location>
    <part>
        <date>2020-12</date>
        <extent unit="page">
            <start>338</start>
            <end>348</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions
%A Ilinykh, Nikolai
%A Dobnik, Simon
%Y Davis, Brian
%Y Graham, Yvette
%Y Kelleher, John
%Y Sripada, Yaji
%S Proceedings of the 13th International Conference on Natural Language Generation
%D 2020
%8 December
%I Association for Computational Linguistics
%C Dublin, Ireland
%F ilinykh-dobnik-2020-image
%X Generating multi-sentence image descriptions is a challenging task, which requires a good model to produce coherent and accurate paragraphs, describing salient objects in the image. We argue that multiple sources of information are beneficial when describing visual scenes with long sequences. These include (i) perceptual information and (ii) semantic (language) information about how to describe what is in the image. We also compare the effects of using two different pooling mechanisms on either a single modality or their combination. We demonstrate that the model which utilises both visual and language inputs can be used to generate accurate and diverse paragraphs when combined with a particular pooling mechanism. The results of our automatic and human evaluation show that learning to embed semantic information along with visual stimuli into the paragraph generation model is not trivial, raising a variety of proposals for future experiments.
%R 10.18653/v1/2020.inlg-1.40
%U https://aclanthology.org/2020.inlg-1.40
%U https://doi.org/10.18653/v1/2020.inlg-1.40
%P 338-348

Markdown (Informal)

[When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions](https://aclanthology.org/2020.inlg-1.40) (Ilinykh & Dobnik, INLG 2020)

• When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions (Ilinykh & Dobnik, INLG 2020)

ACL

• Nikolai Ilinykh and Simon Dobnik. 2020. When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions. In Proceedings of the 13th International Conference on Natural Language Generation, pages 338–348, Dublin, Ireland. Association for Computational Linguistics.