Sherricane Jiayu Xu

Author directory

2026

Answering complex queries over long-context multimodal streams requires dynamically synthesizing asynchronous and heterogeneous evidence across extended temporal and logical spans. Existing approaches often rely on iterative Multimodal Retrieval-Augmented Generation (RAG), facing structural challenges from semantic and modality gaps as well as context bloat induced by multi-step retrieval. To address these challenges, we propose SynapseHSG, a decoupled one-shot grounding framework for long-context multimodal Question Answering. SynapseHSG introduces a hyper-semantic evidence organization method (named M2HSG) that constructs an offline graph index to align cross-modal evidence. During inference, we design a Semantic Decomposer to convert natural-language queries into graph-operable constraints. Guided by these constraints, SynapseHSG retrieves a compact query-anchored subgraph in one shot and linearizes it as structured evidence for a frozen generator. This avoids repeated LLM-driven retrieval while restricting trainable components to graph construction and query parsing. To support systematic evaluation, we introduce MM-ConfQA, a new long-context multimodal QA benchmark built from academic conference presentations. Experimental results on MM-ConfQA show that SynapseHSG improves grounded answer generation over strong multimodal and retrieval baselines, especially on dispersed-evidence questions, highlighting the value of pre-organized multimodal evidence over repeated search in raw heterogeneous contexts. We release the code and data to public.