<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shin-etal-2026-unisaf">
    <titleInfo>
        <title>UNISAF-Bench: Intention-Grounded Safety Evaluation for Unified Multimodal Generation Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yooju</namePart>
        <namePart type="family">Shin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chentao</namePart>
        <namePart type="family">Ye</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Weitong</namePart>
        <namePart type="family">Ruan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rahul</namePart>
        <namePart type="family">Gupta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Unified Multi-Modal Generation (UMMG) models support seamless switching between generation tasks spanning multiple modality combinations within a single conversational interface. However, existing safety benchmarks construct prompts independently per modality, so different modalities cover different harmful intentions, preventing holistic evaluation of UMMG models. We propose an intention-grounded evaluation methodology in which every modality combination is evaluated against the same set of harmful intentions, yielding fair, directly comparable safety scores across modalities. Because scores are aggregated at the intention level — averaging over the same set of intentions rather than over modality-specific prompt sets — each model receives an accurate safety profile not distorted by per-modality differences. The methodology is incrementally extensible: new modality combinations or intentions can be added without rebuilding existing data. Following this methodology, we construct UNISAF-Bench (UNIfied intention-grounded SAFety Benchmark): 1,257 human-verified harmful intentions across 19 policies, yielding 20,112 conversations spanning 4 modality combinations, single-/multi-turn settings, and paired policy-violating/boundary-benign variants. Evaluation of four open-source UMMG models reveals systematic safety gaps: multimodal inputs degrade text-output safety, safety alignment fails to transfer to image generation, and multi-turn deflection bias inflates refusal on both harmful and benign prompts.</abstract>
    <identifier type="citekey">shin-etal-2026-unisaf</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.108/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>2541</start>
            <end>2567</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T UNISAF-Bench: Intention-Grounded Safety Evaluation for Unified Multimodal Generation Models
%A Shin, Yooju
%A Ye, Chentao
%A Ruan, Weitong
%A Gupta, Rahul
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F shin-etal-2026-unisaf
%X Unified Multi-Modal Generation (UMMG) models support seamless switching between generation tasks spanning multiple modality combinations within a single conversational interface. However, existing safety benchmarks construct prompts independently per modality, so different modalities cover different harmful intentions, preventing holistic evaluation of UMMG models. We propose an intention-grounded evaluation methodology in which every modality combination is evaluated against the same set of harmful intentions, yielding fair, directly comparable safety scores across modalities. Because scores are aggregated at the intention level — averaging over the same set of intentions rather than over modality-specific prompt sets — each model receives an accurate safety profile not distorted by per-modality differences. The methodology is incrementally extensible: new modality combinations or intentions can be added without rebuilding existing data. Following this methodology, we construct UNISAF-Bench (UNIfied intention-grounded SAFety Benchmark): 1,257 human-verified harmful intentions across 19 policies, yielding 20,112 conversations spanning 4 modality combinations, single-/multi-turn settings, and paired policy-violating/boundary-benign variants. Evaluation of four open-source UMMG models reveals systematic safety gaps: multimodal inputs degrade text-output safety, safety alignment fails to transfer to image generation, and multi-turn deflection bias inflates refusal on both harmful and benign prompts.
%U https://aclanthology.org/2026.emnlp-main.108/
%P 2541-2567

Markdown (Informal)

[UNISAF-Bench: Intention-Grounded Safety Evaluation for Unified Multimodal Generation Models](https://aclanthology.org/2026.emnlp-main.108/) (Shin et al., EMNLP 2026)

• UNISAF-Bench: Intention-Grounded Safety Evaluation for Unified Multimodal Generation Models (Shin et al., EMNLP 2026)

ACL

• Yooju Shin, Chentao Ye, Weitong Ruan, and Rahul Gupta. 2026. UNISAF-Bench: Intention-Grounded Safety Evaluation for Unified Multimodal Generation Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 2541–2567, Budapest, Hungary. Association for Computational Linguistics.