<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mogrovejo-solorio-2024-question">
    <titleInfo>
        <title>Question-Instructed Visual Descriptions for Zero-Shot Video Answering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">David</namePart>
        <namePart type="family">Mogrovejo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Thamar</namePart>
        <namePart type="family">Solorio</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand and virtual meeting</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present Q-ViD, a simple approach for video question answering (video QA), that unlike prior methods, which are based on complex architectures, computationally expensive pipelines or use closed models like GPTs, Q-ViD relies on a single instruction-aware open vision-language model (InstructBLIP) to tackle videoQA using frame descriptions. Specifically, we create captioning instruction prompts that rely on the target questions about the videos and leverage InstructBLIP to obtain video frame captions that are useful to the task at hand. Subsequently, we form descriptions of the whole video using the question-dependent frame captions, and feed that information, along with a question-answering prompt, to a large language model (LLM). The LLM is our reasoning module, and performs the final step of multiple-choice QA. Our simple Q-ViD framework achieves competitive or even higher performances than current state of the art models on a diverse range of videoQA benchmarks, including NExT-QA, STAR, How2QA, TVQA and IntentQA.</abstract>
    <identifier type="citekey">mogrovejo-solorio-2024-question</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.555</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>9329</start>
            <end>9339</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Question-Instructed Visual Descriptions for Zero-Shot Video Answering
%A Mogrovejo, David
%A Solorio, Thamar
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand and virtual meeting
%F mogrovejo-solorio-2024-question
%X We present Q-ViD, a simple approach for video question answering (video QA), that unlike prior methods, which are based on complex architectures, computationally expensive pipelines or use closed models like GPTs, Q-ViD relies on a single instruction-aware open vision-language model (InstructBLIP) to tackle videoQA using frame descriptions. Specifically, we create captioning instruction prompts that rely on the target questions about the videos and leverage InstructBLIP to obtain video frame captions that are useful to the task at hand. Subsequently, we form descriptions of the whole video using the question-dependent frame captions, and feed that information, along with a question-answering prompt, to a large language model (LLM). The LLM is our reasoning module, and performs the final step of multiple-choice QA. Our simple Q-ViD framework achieves competitive or even higher performances than current state of the art models on a diverse range of videoQA benchmarks, including NExT-QA, STAR, How2QA, TVQA and IntentQA.
%U https://aclanthology.org/2024.findings-acl.555
%P 9329-9339

Markdown (Informal)

[Question-Instructed Visual Descriptions for Zero-Shot Video Answering](https://aclanthology.org/2024.findings-acl.555) (Mogrovejo & Solorio, Findings 2024)

• Question-Instructed Visual Descriptions for Zero-Shot Video Answering (Mogrovejo & Solorio, Findings 2024)

ACL

• David Mogrovejo and Thamar Solorio. 2024. Question-Instructed Visual Descriptions for Zero-Shot Video Answering. In Findings of the Association for Computational Linguistics ACL 2024, pages 9329–9339, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.