Guangming Yao
Author directory2026
SynAV: Integrated Grounding and Captioning for Fine-grained Omni Video Understanding via Balanced Audio-Visual Synergy
Bolun Zhang | Guangming Yao | Canjun Yang | Yi Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Bolun Zhang | Guangming Yao | Canjun Yang | Yi Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Video understanding is rapidly evolving toward omni-modal fusion, yet Multimodal Large Language Models (MLLMs) still struggle with complex audio-visual reasoning and semantic alignment. Current paradigms often rely on independent modal projectors to map features into the LLM space, an architecture that frequently overlooks intrinsic modal heterogeneity and inadvertently reinforces vision-centric bias by avoiding early, direct semantic interaction between native cross-modal representations. To address these challenges, we propose SynAV, a novel MLLM architecture designed for language-grounded multimodal reasoning. Our framework introduces a two-stage hierarchical cross-modal semantic interaction mechanism: it first performs alignment within the textual semantic space, and subsequently executes a visual-to-audio refinement that bridges semantic gaps and captures temporal dynamics. Furthermore, we employ a frame-by-frame interleaved audio-visual token arrangement enriched with explicit textual timestamps. This strategy acts as a temporal-semantic anchoring mechanism, allowing the LLM to better perceive the synchrony between modalities and seamlessly conduct long-sequence logical reasoning. Our model jointly optimizes temporal grounding and dense captioning tasks in an autoregressive manner. Extensive experiments across 11 benchmarks demonstrate that SynAV significantly outperforms state-of-the-art methods, achieving a superior performance in complex audio-visual language reasoning.
MemDreamer: Decoupling Perception and Reasoning for Long Video Understanding via Agentic Hierarchical Graph Memory
Cong Chen | Guo Gan | Kaixiang Ji | Zhaoyang Zhang | Zhen Yang | Guangming Yao | Hao Chen | Jingdong Chen | Yi Yuan | Chunhua Shen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Cong Chen | Guo Gan | Kaixiang Ji | Zhaoyang Zhang | Zhen Yang | Guangming Yao | Hao Chen | Jingdong Chen | Yi Yuan | Chunhua Shen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Current Vision-Language Models struggle with hours-long videos because processing full-length visual sequences induces prohibitive token explosion and attention dilution. To overcome this, we introduce MemDreamer to decouple perception and reasoning, shifting long-video understanding into an agentic exploration process. As a plug-and-play framework, it incrementally streams videos to construct a Hierarchical Graph Memory, a top-down three-tier architecture for semantic abstraction, anchored by a foundational graph capturing spatiotemporal and causal relations. During inference, the reasoning model employs agentic tool-augmented retrieval, navigating hierarchies, searching nodes, and traversing logical edges via an Observation-Reason-Action loop. Experiments show MemDreamer achieves SOTA results across four mainstream benchmarks, substantially narrowing the gap with human experts. It greatly reduces the reasoning context relative to full-context ingestion while delivering a substantial absolute accuracy gain. Furthermore, statistical analysis uncovers a strong positive linear correlation between an VLM’s performance on logic reasoning and long-video understanding benchmarks, establishing agentic capability scaling as a new paradigm for multimodal comprehension.