<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ge-etal-2023-wrong">
    <titleInfo>
        <title>From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jiaxin</namePart>
        <namePart type="family">Ge</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sanjay</namePart>
        <namePart type="family">Subramanian</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Trevor</namePart>
        <namePart type="family">Darrell</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Boyi</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Addressing the challenge of adapting pre-trained vision-language models for generating insightful explanations for visual reasoning tasks with limited annotations, we present ReVisE: a Recursive Visual Explanation algorithm. Our method iteratively computes visual features (conditioned on the text input), an answer, and an explanation, to improve the explanation quality step by step until the answer converges. We find that this multi-step approach guides the model to correct its own answers and outperforms single-step explanation generation. Furthermore, explanations generated by ReVisE also serve as valuable annotations for few-shot self-training. Our approach outperforms previous methods while utilizing merely 5% of the human-annotated explanations across 10 metrics, demonstrating up to a 4.2 and 1.3 increase in BLEU-1 score on the VCR and VQA-X datasets, underscoring the efficacy and data-efficiency of our method.</abstract>
    <identifier type="citekey">ge-etal-2023-wrong</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.75</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.75</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>1173</start>
            <end>1185</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation
%A Ge, Jiaxin
%A Subramanian, Sanjay
%A Darrell, Trevor
%A Li, Boyi
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F ge-etal-2023-wrong
%X Addressing the challenge of adapting pre-trained vision-language models for generating insightful explanations for visual reasoning tasks with limited annotations, we present ReVisE: a Recursive Visual Explanation algorithm. Our method iteratively computes visual features (conditioned on the text input), an answer, and an explanation, to improve the explanation quality step by step until the answer converges. We find that this multi-step approach guides the model to correct its own answers and outperforms single-step explanation generation. Furthermore, explanations generated by ReVisE also serve as valuable annotations for few-shot self-training. Our approach outperforms previous methods while utilizing merely 5% of the human-annotated explanations across 10 metrics, demonstrating up to a 4.2 and 1.3 increase in BLEU-1 score on the VCR and VQA-X datasets, underscoring the efficacy and data-efficiency of our method.
%R 10.18653/v1/2023.emnlp-main.75
%U https://aclanthology.org/2023.emnlp-main.75
%U https://doi.org/10.18653/v1/2023.emnlp-main.75
%P 1173-1185

Markdown (Informal)

[From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation](https://aclanthology.org/2023.emnlp-main.75) (Ge et al., EMNLP 2023)

• From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation (Ge et al., EMNLP 2023)

ACL

• Jiaxin Ge, Sanjay Subramanian, Trevor Darrell, and Boyi Li. 2023. From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1173–1185, Singapore. Association for Computational Linguistics.