<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhang-etal-2026-synav">
    <titleInfo>
        <title>SynAV: Integrated Grounding and Captioning for Fine-grained Omni Video Understanding via Balanced Audio-Visual Synergy</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Bolun</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Guangming</namePart>
        <namePart type="family">Yao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Canjun</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yi</namePart>
        <namePart type="family">Yuan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>Video understanding is rapidly evolving toward omni-modal fusion, yet Multimodal Large Language Models (MLLMs) still struggle with complex audio-visual reasoning and semantic alignment. Current paradigms often rely on independent modal projectors to map features into the LLM space, an architecture that frequently overlooks intrinsic modal heterogeneity and inadvertently reinforces vision-centric bias by avoiding early, direct semantic interaction between native cross-modal representations. To address these challenges, we propose SynAV, a novel MLLM architecture designed for language-grounded multimodal reasoning. Our framework introduces a two-stage hierarchical cross-modal semantic interaction mechanism: it first performs alignment within the textual semantic space, and subsequently executes a visual-to-audio refinement that bridges semantic gaps and captures temporal dynamics. Furthermore, we employ a frame-by-frame interleaved audio-visual token arrangement enriched with explicit textual timestamps. This strategy acts as a temporal-semantic anchoring mechanism, allowing the LLM to better perceive the synchrony between modalities and seamlessly conduct long-sequence logical reasoning. Our model jointly optimizes temporal grounding and dense captioning tasks in an autoregressive manner. Extensive experiments across 11 benchmarks demonstrate that SynAV significantly outperforms state-of-the-art methods, achieving a superior performance in complex audio-visual language reasoning.</abstract>
    <identifier type="citekey">zhang-etal-2026-synav</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.581/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>12259</start>
            <end>12282</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T SynAV: Integrated Grounding and Captioning for Fine-grained Omni Video Understanding via Balanced Audio-Visual Synergy
%A Zhang, Bolun
%A Yao, Guangming
%A Yang, Canjun
%A Yuan, Yi
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F zhang-etal-2026-synav
%X Video understanding is rapidly evolving toward omni-modal fusion, yet Multimodal Large Language Models (MLLMs) still struggle with complex audio-visual reasoning and semantic alignment. Current paradigms often rely on independent modal projectors to map features into the LLM space, an architecture that frequently overlooks intrinsic modal heterogeneity and inadvertently reinforces vision-centric bias by avoiding early, direct semantic interaction between native cross-modal representations. To address these challenges, we propose SynAV, a novel MLLM architecture designed for language-grounded multimodal reasoning. Our framework introduces a two-stage hierarchical cross-modal semantic interaction mechanism: it first performs alignment within the textual semantic space, and subsequently executes a visual-to-audio refinement that bridges semantic gaps and captures temporal dynamics. Furthermore, we employ a frame-by-frame interleaved audio-visual token arrangement enriched with explicit textual timestamps. This strategy acts as a temporal-semantic anchoring mechanism, allowing the LLM to better perceive the synchrony between modalities and seamlessly conduct long-sequence logical reasoning. Our model jointly optimizes temporal grounding and dense captioning tasks in an autoregressive manner. Extensive experiments across 11 benchmarks demonstrate that SynAV significantly outperforms state-of-the-art methods, achieving a superior performance in complex audio-visual language reasoning.
%U https://aclanthology.org/2026.findings-emnlp.581/
%P 12259-12282

Markdown (Informal)

[SynAV: Integrated Grounding and Captioning for Fine-grained Omni Video Understanding via Balanced Audio-Visual Synergy](https://aclanthology.org/2026.findings-emnlp.581/) (Zhang et al., Findings 2026)

• SynAV: Integrated Grounding and Captioning for Fine-grained Omni Video Understanding via Balanced Audio-Visual Synergy (Zhang et al., Findings 2026)

ACL

• Bolun Zhang, Guangming Yao, Canjun Yang, and Yi Yuan. 2026. SynAV: Integrated Grounding and Captioning for Fine-grained Omni Video Understanding via Balanced Audio-Visual Synergy. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 12259–12282, Budapest, Hungary. Association for Computational Linguistics.