<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="he-etal-2026-beyond">
    <titleInfo>
        <title>Beyond Referring Expressions: Scenario Comprehension Visual Grounding</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ruozhen</namePart>
        <namePart type="family">He</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nisarg</namePart>
        <namePart type="given">A</namePart>
        <namePart type="family">Shah</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Qihua</namePart>
        <namePart type="family">Dong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zilin</namePart>
        <namePart type="family">Xiao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jaywon</namePart>
        <namePart type="family">Koo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vicente</namePart>
        <namePart type="family">Ordonez</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Existing visual grounding benchmarks primarily evaluate alignment between image regions and literal referring expressions, where models can often succeed by matching a prominent named category. This leaves a complementary but under-examined setting: scenario-based visual grounding, where the target must be inferred from roles, intentions, and relational context rather than explicit object naming alone. We introduce Referring Scenario Comprehension (RSC), a benchmark designed to study this setting. RSC queries describe roles, goals, and contextual cues, and deliberately reference distractor objects to reduce lexical shortcutting. Each instance is annotated with interpretable difficulty tags for uniqueness, clutter, size, overlap, and position, which expose distinct failure modes and support fine-grained analysis. RSC contains approximately 31k training examples, 4k in-domain test examples, and a 3k out-of-distribution split with unseen object categories to test generalization beyond category co-occurrence. We further propose ScenGround, a curriculum reasoning method serving as a reference point for this setting, combining supervised warm-starting with difficulty-aware reinforcement learning. Experiments show that scenario-based queries expose systematic shortcut failures in current models that standard benchmarks do not reveal, and that curriculum training improves performance on challenging slices and transfers robust improvement to standard benchmarks.</abstract>
    <identifier type="citekey">he-etal-2026-beyond</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1788/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>40742</start>
            <end>40762</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Beyond Referring Expressions: Scenario Comprehension Visual Grounding
%A He, Ruozhen
%A Shah, Nisarg A.
%A Dong, Qihua
%A Xiao, Zilin
%A Koo, Jaywon
%A Ordonez, Vicente
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F he-etal-2026-beyond
%X Existing visual grounding benchmarks primarily evaluate alignment between image regions and literal referring expressions, where models can often succeed by matching a prominent named category. This leaves a complementary but under-examined setting: scenario-based visual grounding, where the target must be inferred from roles, intentions, and relational context rather than explicit object naming alone. We introduce Referring Scenario Comprehension (RSC), a benchmark designed to study this setting. RSC queries describe roles, goals, and contextual cues, and deliberately reference distractor objects to reduce lexical shortcutting. Each instance is annotated with interpretable difficulty tags for uniqueness, clutter, size, overlap, and position, which expose distinct failure modes and support fine-grained analysis. RSC contains approximately 31k training examples, 4k in-domain test examples, and a 3k out-of-distribution split with unseen object categories to test generalization beyond category co-occurrence. We further propose ScenGround, a curriculum reasoning method serving as a reference point for this setting, combining supervised warm-starting with difficulty-aware reinforcement learning. Experiments show that scenario-based queries expose systematic shortcut failures in current models that standard benchmarks do not reveal, and that curriculum training improves performance on challenging slices and transfers robust improvement to standard benchmarks.
%U https://aclanthology.org/2026.emnlp-main.1788/
%P 40742-40762

Markdown (Informal)

[Beyond Referring Expressions: Scenario Comprehension Visual Grounding](https://aclanthology.org/2026.emnlp-main.1788/) (He et al., EMNLP 2026)

• Beyond Referring Expressions: Scenario Comprehension Visual Grounding (He et al., EMNLP 2026)

ACL

• Ruozhen He, Nisarg A Shah, Qihua Dong, Zilin Xiao, Jaywon Koo, and Vicente Ordonez. 2026. Beyond Referring Expressions: Scenario Comprehension Visual Grounding. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 40742–40762, Budapest, Hungary. Association for Computational Linguistics.