<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="junker-zarriess-2024-resilience-scene">
    <titleInfo>
        <title>Resilience through Scene Context in Visual Referring Expression Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Simeon</namePart>
        <namePart type="family">Junker</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sina</namePart>
        <namePart type="family">Zarrieß</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 17th International Natural Language Generation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Saad</namePart>
            <namePart type="family">Mahamood</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nguyen</namePart>
            <namePart type="given">Le</namePart>
            <namePart type="family">Minh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Daphne</namePart>
            <namePart type="family">Ippolito</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Tokyo, Japan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Scene context is well known to facilitate humans’ perception of visible objects. In this paper, we investigate the role of context in Referring Expression Generation (REG) for objects in images, where existing research has often focused on distractor contexts that exert pressure on the generator. We take a new perspective on scene context in REG and hypothesize that contextual information can be conceived of as a resource that makes REG models more resilient and facilitates the generation of object descriptions, and object types in particular. We train and test Transformer-based REG models with target representations that have been artificially obscured with noise to varying degrees. We evaluate how properties of the models’ visual context affect their processing and performance. Our results show that even simple scene contexts make models surprisingly resilient to perturbations, to the extent that they can identify referent types even when visual information about the target is completely missing.</abstract>
    <identifier type="citekey">junker-zarriess-2024-resilience-scene</identifier>
    <location>
        <url>https://aclanthology.org/2024.inlg-main.29</url>
    </location>
    <part>
        <date>2024-09</date>
        <extent unit="page">
            <start>344</start>
            <end>357</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Resilience through Scene Context in Visual Referring Expression Generation
%A Junker, Simeon
%A Zarrieß, Sina
%Y Mahamood, Saad
%Y Minh, Nguyen Le
%Y Ippolito, Daphne
%S Proceedings of the 17th International Natural Language Generation Conference
%D 2024
%8 September
%I Association for Computational Linguistics
%C Tokyo, Japan
%F junker-zarriess-2024-resilience-scene
%X Scene context is well known to facilitate humans’ perception of visible objects. In this paper, we investigate the role of context in Referring Expression Generation (REG) for objects in images, where existing research has often focused on distractor contexts that exert pressure on the generator. We take a new perspective on scene context in REG and hypothesize that contextual information can be conceived of as a resource that makes REG models more resilient and facilitates the generation of object descriptions, and object types in particular. We train and test Transformer-based REG models with target representations that have been artificially obscured with noise to varying degrees. We evaluate how properties of the models’ visual context affect their processing and performance. Our results show that even simple scene contexts make models surprisingly resilient to perturbations, to the extent that they can identify referent types even when visual information about the target is completely missing.
%U https://aclanthology.org/2024.inlg-main.29
%P 344-357

Markdown (Informal)

[Resilience through Scene Context in Visual Referring Expression Generation](https://aclanthology.org/2024.inlg-main.29) (Junker & Zarrieß, INLG 2024)

• Resilience through Scene Context in Visual Referring Expression Generation (Junker & Zarrieß, INLG 2024)

ACL

• Simeon Junker and Sina Zarrieß. 2024. Resilience through Scene Context in Visual Referring Expression Generation. In Proceedings of the 17th International Natural Language Generation Conference, pages 344–357, Tokyo, Japan. Association for Computational Linguistics.