<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shekhar-etal-2019-evaluating">
    <titleInfo>
        <title>Evaluating the Representational Hub of Language and Vision Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ravi</namePart>
        <namePart type="family">Shekhar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ece</namePart>
        <namePart type="family">Takmaz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Raquel</namePart>
        <namePart type="family">Fernández</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Raffaella</namePart>
        <namePart type="family">Bernardi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 13th International Conference on Computational Semantics - Long Papers</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Dobnik</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Stergios</namePart>
            <namePart type="family">Chatzikyriakidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vera</namePart>
            <namePart type="family">Demberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gothenburg, Sweden</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The multimodal models used in the emerging field at the intersection of computational linguistics and computer vision implement the bottom-up processing of the “Hub and Spoke” architecture proposed in cognitive science to represent how the brain processes and combines multi-sensory inputs. In particular, the Hub is implemented as a neural network encoder. We investigate the effect on this encoder of various vision-and-language tasks proposed in the literature: visual question answering, visual reference resolution, and visually grounded dialogue. To measure the quality of the representations learned by the encoder, we use two kinds of analyses. First, we evaluate the encoder pre-trained on the different vision-and-language tasks on an existing “diagnostic task” designed to assess multimodal semantic understanding. Second, we carry out a battery of analyses aimed at studying how the encoder merges and exploits the two modalities.</abstract>
    <identifier type="citekey">shekhar-etal-2019-evaluating</identifier>
    <identifier type="doi">10.18653/v1/W19-0418</identifier>
    <location>
        <url>https://aclanthology.org/W19-0418</url>
    </location>
    <part>
        <date>2019-05</date>
        <extent unit="page">
            <start>211</start>
            <end>222</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Evaluating the Representational Hub of Language and Vision Models
%A Shekhar, Ravi
%A Takmaz, Ece
%A Fernández, Raquel
%A Bernardi, Raffaella
%Y Dobnik, Simon
%Y Chatzikyriakidis, Stergios
%Y Demberg, Vera
%S Proceedings of the 13th International Conference on Computational Semantics - Long Papers
%D 2019
%8 May
%I Association for Computational Linguistics
%C Gothenburg, Sweden
%F shekhar-etal-2019-evaluating
%X The multimodal models used in the emerging field at the intersection of computational linguistics and computer vision implement the bottom-up processing of the “Hub and Spoke” architecture proposed in cognitive science to represent how the brain processes and combines multi-sensory inputs. In particular, the Hub is implemented as a neural network encoder. We investigate the effect on this encoder of various vision-and-language tasks proposed in the literature: visual question answering, visual reference resolution, and visually grounded dialogue. To measure the quality of the representations learned by the encoder, we use two kinds of analyses. First, we evaluate the encoder pre-trained on the different vision-and-language tasks on an existing “diagnostic task” designed to assess multimodal semantic understanding. Second, we carry out a battery of analyses aimed at studying how the encoder merges and exploits the two modalities.
%R 10.18653/v1/W19-0418
%U https://aclanthology.org/W19-0418
%U https://doi.org/10.18653/v1/W19-0418
%P 211-222

Markdown (Informal)

[Evaluating the Representational Hub of Language and Vision Models](https://aclanthology.org/W19-0418) (Shekhar et al., IWCS 2019)

• Evaluating the Representational Hub of Language and Vision Models (Shekhar et al., IWCS 2019)

ACL

• Ravi Shekhar, Ece Takmaz, Raquel Fernández, and Raffaella Bernardi. 2019. Evaluating the Representational Hub of Language and Vision Models. In Proceedings of the 13th International Conference on Computational Semantics - Long Papers, pages 211–222, Gothenburg, Sweden. Association for Computational Linguistics.