<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wu-mooney-2022-entity">
    <titleInfo>
        <title>Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jialin</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Raymond</namePart>
        <namePart type="family">Mooney</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Most Outside-Knowledge Visual Question Answering (OK-VQA) systems employ a two-stage framework that first retrieves external knowledge given the visual question and then predicts the answer based on the retrieved content. However, the retrieved knowledge is often inadequate. Retrievals are frequently too general and fail to cover specific knowledge needed to answer the question. Also, the naturally available supervision (whether the passage contains the correct answer) is weak and does not guarantee question relevancy. To address these issues, we propose an Entity-Focused Retrieval (EnFoRe) model that provides stronger supervision during training and recognizes question-relevant entities to help retrieve more specific knowledge. Experiments show that our EnFoRe model achieves superior retrieval performance on OK-VQA, the currently largest outside-knowledge VQA dataset. We also combine the retrieved knowledge with state-of-the-art VQA models, and achieve a new state-of-the-art performance on OK-VQA.</abstract>
    <identifier type="citekey">wu-mooney-2022-entity</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.551</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>8061</start>
            <end>8072</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering
%A Wu, Jialin
%A Mooney, Raymond
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F wu-mooney-2022-entity
%X Most Outside-Knowledge Visual Question Answering (OK-VQA) systems employ a two-stage framework that first retrieves external knowledge given the visual question and then predicts the answer based on the retrieved content. However, the retrieved knowledge is often inadequate. Retrievals are frequently too general and fail to cover specific knowledge needed to answer the question. Also, the naturally available supervision (whether the passage contains the correct answer) is weak and does not guarantee question relevancy. To address these issues, we propose an Entity-Focused Retrieval (EnFoRe) model that provides stronger supervision during training and recognizes question-relevant entities to help retrieve more specific knowledge. Experiments show that our EnFoRe model achieves superior retrieval performance on OK-VQA, the currently largest outside-knowledge VQA dataset. We also combine the retrieved knowledge with state-of-the-art VQA models, and achieve a new state-of-the-art performance on OK-VQA.
%U https://aclanthology.org/2022.emnlp-main.551
%P 8061-8072

Markdown (Informal)

[Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering](https://aclanthology.org/2022.emnlp-main.551) (Wu & Mooney, EMNLP 2022)

• Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering (Wu & Mooney, EMNLP 2022)

ACL

• Jialin Wu and Raymond Mooney. 2022. Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8061–8072, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.