AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention

Venkatesh Velugubantla; Raj Sonani; Msvpj Sathvik

doi:10.18653/v1/2025.nlp4pi-1.25

AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention

Venkatesh Velugubantla, Raj Sonani, Msvpj Sathvik

Correct Metadata for

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

Advanced AI models that generate realistic images from text prompts offer new creative possibilities but also risk producing culturally insensitive or offensive content. To address this issue, we introduce a novel dataset designed to classify text prompts that could lead to the generation of harmful images misrepresenting different cultures and communities. By training machine learning models on this dataset, we aim to automatically identify and filter out harmful prompts before image generation, balancing cultural sensitivity with creative freedom. Benchmarking with state-ofthe-art language models, our baseline models achieved an accuracy of 73.34%.

Anthology ID:: 2025.nlp4pi-1.25
Volume:: Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI)
Month:: July
Year:: 2025
Address:: Vienna, Austria
Editors:: Katherine Atwell, Laura Biester, Angana Borah, Daryna Dementieva, Oana Ignat, Neema Kotonya, Ziyi Liu, Ruyuan Wan, Steven Wilson, Jieyu Zhao
Venues:: NLP4PI | WS
SIG:
Publisher:: Association for Computational Linguistics
Note:
Pages:: 285–293
Language:
URL:: https://aclanthology.org/2025.nlp4pi-1.25/
DOI:: 10.18653/v1/2025.nlp4pi-1.25
Bibkey:
Cite (ACL):: Venkatesh Velugubantla, Raj Sonani, and Msvpj Sathvik. 2025. AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention. In Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI), pages 285–293, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):: AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention (Velugubantla et al., NLP4PI 2025)
Copy Citation:
PDF:: https://aclanthology.org/2025.nlp4pi-1.25.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{velugubantla-etal-2025-ai,
    title = "{AI} Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention",
    author = "Velugubantla, Venkatesh  and
      Sonani, Raj  and
      Sathvik, Msvpj",
    editor = "Atwell, Katherine  and
      Biester, Laura  and
      Borah, Angana  and
      Dementieva, Daryna  and
      Ignat, Oana  and
      Kotonya, Neema  and
      Liu, Ziyi  and
      Wan, Ruyuan  and
      Wilson, Steven  and
      Zhao, Jieyu",
    booktitle = "Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.nlp4pi-1.25/",
    doi = "10.18653/v1/2025.nlp4pi-1.25",
    pages = "285--293",
    ISBN = "978-1-959429-19-7",
    abstract = "Advanced AI models that generate realistic images from text prompts offer new creative possibilities but also risk producing culturally insensitive or offensive content. To address this issue, we introduce a novel dataset designed to classify text prompts that could lead to the generation of harmful images misrepresenting different cultures and communities. By training machine learning models on this dataset, we aim to automatically identify and filter out harmful prompts before image generation, balancing cultural sensitivity with creative freedom. Benchmarking with state-ofthe-art language models, our baseline models achieved an accuracy of 73.34{\%}."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="velugubantla-etal-2025-ai">
    <titleInfo>
        <title>AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Venkatesh</namePart>
        <namePart type="family">Velugubantla</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Raj</namePart>
        <namePart type="family">Sonani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Msvpj</namePart>
        <namePart type="family">Sathvik</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Katherine</namePart>
            <namePart type="family">Atwell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laura</namePart>
            <namePart type="family">Biester</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Angana</namePart>
            <namePart type="family">Borah</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Daryna</namePart>
            <namePart type="family">Dementieva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Oana</namePart>
            <namePart type="family">Ignat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Neema</namePart>
            <namePart type="family">Kotonya</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ziyi</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruyuan</namePart>
            <namePart type="family">Wan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Steven</namePart>
            <namePart type="family">Wilson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jieyu</namePart>
            <namePart type="family">Zhao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vienna, Austria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">978-1-959429-19-7</identifier>
    </relatedItem>
    <abstract>Advanced AI models that generate realistic images from text prompts offer new creative possibilities but also risk producing culturally insensitive or offensive content. To address this issue, we introduce a novel dataset designed to classify text prompts that could lead to the generation of harmful images misrepresenting different cultures and communities. By training machine learning models on this dataset, we aim to automatically identify and filter out harmful prompts before image generation, balancing cultural sensitivity with creative freedom. Benchmarking with state-ofthe-art language models, our baseline models achieved an accuracy of 73.34%.</abstract>
    <identifier type="citekey">velugubantla-etal-2025-ai</identifier>
    <identifier type="doi">10.18653/v1/2025.nlp4pi-1.25</identifier>
    <location>
        <url>https://aclanthology.org/2025.nlp4pi-1.25/</url>
    </location>
    <part>
        <date>2025-07</date>
        <extent unit="page">
            <start>285</start>
            <end>293</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention
%A Velugubantla, Venkatesh
%A Sonani, Raj
%A Sathvik, Msvpj
%Y Atwell, Katherine
%Y Biester, Laura
%Y Borah, Angana
%Y Dementieva, Daryna
%Y Ignat, Oana
%Y Kotonya, Neema
%Y Liu, Ziyi
%Y Wan, Ruyuan
%Y Wilson, Steven
%Y Zhao, Jieyu
%S Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI)
%D 2025
%8 July
%I Association for Computational Linguistics
%C Vienna, Austria
%@ 978-1-959429-19-7
%F velugubantla-etal-2025-ai
%X Advanced AI models that generate realistic images from text prompts offer new creative possibilities but also risk producing culturally insensitive or offensive content. To address this issue, we introduce a novel dataset designed to classify text prompts that could lead to the generation of harmful images misrepresenting different cultures and communities. By training machine learning models on this dataset, we aim to automatically identify and filter out harmful prompts before image generation, balancing cultural sensitivity with creative freedom. Benchmarking with state-ofthe-art language models, our baseline models achieved an accuracy of 73.34%.
%R 10.18653/v1/2025.nlp4pi-1.25
%U https://aclanthology.org/2025.nlp4pi-1.25/
%U https://doi.org/10.18653/v1/2025.nlp4pi-1.25
%P 285-293

Download as File

Markdown (Informal)

[AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention](https://aclanthology.org/2025.nlp4pi-1.25/) (Velugubantla et al., NLP4PI 2025)

AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention (Velugubantla et al., NLP4PI 2025)

ACL

Venkatesh Velugubantla, Raj Sonani, and Msvpj Sathvik. 2025. AI Tools Can Generate Misculture Visuals! Detecting Prompts Generating Misculture Visuals For Prevention. In Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI), pages 285–293, Vienna, Austria. Association for Computational Linguistics.