<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mahendru-etal-2017-promise">
    <titleInfo>
        <title>The Promise of Premise: Harnessing Question Premises in Visual Question Answering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Aroma</namePart>
        <namePart type="family">Mahendru</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Viraj</namePart>
        <namePart type="family">Prabhu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Akrit</namePart>
        <namePart type="family">Mohapatra</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dhruv</namePart>
        <namePart type="family">Batra</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Stefan</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Martha</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rebecca</namePart>
            <namePart type="family">Hwa</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sebastian</namePart>
            <namePart type="family">Riedel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we make a simple observation that questions about images often contain premises – objects and relationships implied by the question – and that reasoning about premises can help Visual Question Answering (VQA) models respond more intelligently to irrelevant or previously unseen questions. When presented with a question that is irrelevant to an image, state-of-the-art VQA models will still answer purely based on learned language biases, resulting in non-sensical or even misleading answers. We note that a visual question is irrelevant to an image if at least one of its premises is false (i.e. not depicted in the image). We leverage this observation to construct a dataset for Question Relevance Prediction and Explanation (QRPE) by searching for false premises. We train novel question relevance detection models and show that models that reason about premises consistently outperform models that do not. We also find that forcing standard VQA models to reason about premises during training can lead to improvements on tasks requiring compositional reasoning.</abstract>
    <identifier type="citekey">mahendru-etal-2017-promise</identifier>
    <identifier type="doi">10.18653/v1/D17-1097</identifier>
    <location>
        <url>https://aclanthology.org/D17-1097</url>
    </location>
    <part>
        <date>2017-09</date>
        <extent unit="page">
            <start>926</start>
            <end>935</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Promise of Premise: Harnessing Question Premises in Visual Question Answering
%A Mahendru, Aroma
%A Prabhu, Viraj
%A Mohapatra, Akrit
%A Batra, Dhruv
%A Lee, Stefan
%Y Palmer, Martha
%Y Hwa, Rebecca
%Y Riedel, Sebastian
%S Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
%D 2017
%8 September
%I Association for Computational Linguistics
%C Copenhagen, Denmark
%F mahendru-etal-2017-promise
%X In this paper, we make a simple observation that questions about images often contain premises – objects and relationships implied by the question – and that reasoning about premises can help Visual Question Answering (VQA) models respond more intelligently to irrelevant or previously unseen questions. When presented with a question that is irrelevant to an image, state-of-the-art VQA models will still answer purely based on learned language biases, resulting in non-sensical or even misleading answers. We note that a visual question is irrelevant to an image if at least one of its premises is false (i.e. not depicted in the image). We leverage this observation to construct a dataset for Question Relevance Prediction and Explanation (QRPE) by searching for false premises. We train novel question relevance detection models and show that models that reason about premises consistently outperform models that do not. We also find that forcing standard VQA models to reason about premises during training can lead to improvements on tasks requiring compositional reasoning.
%R 10.18653/v1/D17-1097
%U https://aclanthology.org/D17-1097
%U https://doi.org/10.18653/v1/D17-1097
%P 926-935

Markdown (Informal)

[The Promise of Premise: Harnessing Question Premises in Visual Question Answering](https://aclanthology.org/D17-1097) (Mahendru et al., EMNLP 2017)

• The Promise of Premise: Harnessing Question Premises in Visual Question Answering (Mahendru et al., EMNLP 2017)

ACL

• Aroma Mahendru, Viraj Prabhu, Akrit Mohapatra, Dhruv Batra, and Stefan Lee. 2017. The Promise of Premise: Harnessing Question Premises in Visual Question Answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 926–935, Copenhagen, Denmark. Association for Computational Linguistics.