<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hirota-etal-2024-descriptive">
    <titleInfo>
        <title>From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yusuke</namePart>
        <namePart type="family">Hirota</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ryo</namePart>
        <namePart type="family">Hachiuma</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chao-Han</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yuta</namePart>
        <namePart type="family">Nakashima</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large language models (LLMs) have enhanced the capacity of vision-language models to caption visual text. This generative approach to image caption enrichment further makes textual captions more descriptive, improving alignment with the visual context. However, while many studies focus on the benefits of generative caption enrichment (GCE), are there any negative side effects? We compare standard-format captions and recent GCE processes from the perspectives of gender bias and hallucination, showing that enriched captions suffer from increased gender bias and hallucination. Furthermore, models trained on these enriched captions amplify gender bias by an average of 30.9% and increase hallucination by 59.5%. This study serves as a caution against the trend of making captions more descriptive.</abstract>
    <identifier type="citekey">hirota-etal-2024-descriptive</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.986</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>17807</start>
            <end>17816</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment
%A Hirota, Yusuke
%A Hachiuma, Ryo
%A Yang, Chao-Han
%A Nakashima, Yuta
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F hirota-etal-2024-descriptive
%X Large language models (LLMs) have enhanced the capacity of vision-language models to caption visual text. This generative approach to image caption enrichment further makes textual captions more descriptive, improving alignment with the visual context. However, while many studies focus on the benefits of generative caption enrichment (GCE), are there any negative side effects? We compare standard-format captions and recent GCE processes from the perspectives of gender bias and hallucination, showing that enriched captions suffer from increased gender bias and hallucination. Furthermore, models trained on these enriched captions amplify gender bias by an average of 30.9% and increase hallucination by 59.5%. This study serves as a caution against the trend of making captions more descriptive.
%U https://aclanthology.org/2024.emnlp-main.986
%P 17807-17816

Markdown (Informal)

[From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment](https://aclanthology.org/2024.emnlp-main.986) (Hirota et al., EMNLP 2024)

• From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment (Hirota et al., EMNLP 2024)

ACL

• Yusuke Hirota, Ryo Hachiuma, Chao-Han Yang, and Yuta Nakashima. 2024. From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17807–17816, Miami, Florida, USA. Association for Computational Linguistics.