Zijie Zhang
Author directoryOther people with similar names: Zijie Zhang
Unverified author pages with similar names: Zijie Zhang
2026
OmniHalluc-L: Counterfactual Benchmarking and Modality-Perturbation Calibration for Long-Form Omni Hallucination
Zixuan Dong | Jiafu Tang | Zhide Lei | Zhe Cao | Zijie Zhang | Yanghai Wang | Shihao Li | Xiaodong Wang | Baoyun Peng | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zixuan Dong | Jiafu Tang | Zhide Lei | Zhe Cao | Zijie Zhang | Yanghai Wang | Shihao Li | Xiaodong Wang | Baoyun Peng | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Long-video Omni assistants often fail not by inventing content, but by misbinding real evidence: they hear the right utterance and see the right event, yet attach it to the wrong speaker, moment, or modality. These almost-true errors evade standard video QA because local evidence remains valid, so item-level scoring can reward both a supported claim and its near-counterfactual. We introduce a counterfactual event-binding protocol that constructs paired supported/counterfactual claims from the same audio-visual event evidence and evaluates them by strict-pair accuracy. We instantiate it as OmniHalluc-L, to our knowledge the first benchmark for long-video Omni hallucination, with 3600 single-claim QA items from 638 long-form videos averaging 24.16 minutes and covering 256.87 hours. Under this protocol, open-weight Omni models remain weak at pair-level binding: Qwen2.5-Omni-7B reaches 32.06% and Qwen3-Omni-Instruct reaches 41.55%, versus 76.54% for a closed-source reference. To narrow this gap, we propose MPRC (Modality-Perturbation Reliability Calibration), a frozen-backbone calibrator that combines native confidence with response shifts from selected audio-negative probes. MPRC raises these two models to 36.22% and 51.09%, respectively, and improves target-adapted MCQ accuracy on OmniVideoBench and WorldSense. The code is available at https://github.com/ZexDong/omnihallucL-mprc.
DR3-Eval: Towards Realistic and Reproducible Deep Research Evaluation
Qianqian Xie | He Zhu | Xueming Han | Qingheng Xiong | Tiantian Xia | Fanyu Meng | Jiakai Wang | Zhiqi Bai | Chengkang Jiang | Zhaohui Wang | Yubin Guo | Yuqing Wen | Jiayang Mao | Zijie Zhang | Shihao Li | Yanghai Wang | Yuxiang Ren | Junlan Feng | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Qianqian Xie | He Zhu | Xueming Han | Qingheng Xiong | Tiantian Xia | Fanyu Meng | Jiakai Wang | Zhiqi Bai | Chengkang Jiang | Zhaohui Wang | Yubin Guo | Yuqing Wen | Jiayang Mao | Zijie Zhang | Shihao Li | Yanghai Wang | Yuxiang Ren | Junlan Feng | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Deep Research Agents (DRAs) aim to solve complex, long-horizon research tasks involving planning, retrieval, multimodal understanding, and report generation, yet their evaluation remains challenging due to dynamic web environments and ambiguous task definitions. We propose **DR3-Eval**, a realistic and reproducible benchmark for evaluating deep research agents on multimodal, multi-file report generation. DR3-Eval is constructed from authentic user-provided materials and paired with a per-task static research sandbox corpus that simulates open-web complexity while remaining fully verifiable, containing supportive documents, distractors, and noise. Moreover, we introduce a multi-dimensional evaluation framework measuring Information Recall, Factual Accuracy, Citation Coverage, Instruction Following, and Depth Quality, and validate its alignment with human judgments. Experiments with our developed multi-agent system DR3-Agent based on multiple state-of-the-art language models demonstrate that DR3-Eval is highly challenging and reveals critical failure modes in retrieval robustness and hallucination control.
AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning
Yanghai Wang | Jiahao Wang | Jiafu Tang | Yuanxing Zhang | Zhe Cao | Hanyan Bian | Zijie Zhang | Weiliang Luo | Zhiyu Pan | Zixuan Dong | Jiaheng Liu | Zhaoxiang Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yanghai Wang | Jiahao Wang | Jiafu Tang | Yuanxing Zhang | Zhe Cao | Hanyan Bian | Zijie Zhang | Weiliang Luo | Zhiyu Pan | Zixuan Dong | Jiaheng Liu | Zhaoxiang Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Omni-modal video captioning is not merely combining visual captioning with audio transcription: a useful caption must describe how visual actions, speech, music, and sound effects co-evolve over time. Existing large multimodal models often fail at this relational step, treating audio and visual streams as loosely coupled observations, relying heavily on automatic speech recognition, and under-specifying non-speech sounds and their links to visual events. We present AVSCap, a framework for audio-visual captioning centered on explicit cross-modal event binding. First, we construct AVSCap-130K, a tri-modal training corpus generated by a decoupled-then-fused pipeline that anchors visual and acoustic evidence independently before composing temporally grounded omni-modal captions. Second, we train AVSCap-7B, a 7B captioner with a two-stage strategy: supervised fine-tuning establishes baseline captioning ability, while a sample-efficient reinforcement learning stage uses hybrid rewards to optimize acoustic completeness and audio-visual synergy. Our scaling analysis shows that reinforcement learning brings larger gains than simply increasing SFT data. Third, we introduce AVSCapBench, a human-curated benchmark that decomposes captions into visual, audio, and synergy events and evaluates them with fine-grained event recall. Experiments on AVSCapBench and external benchmarks show that AVSCap-7B improves non-speech audio coverage and cross-modal binding, delivering the best overall performance among evaluated open-source models.
Search
Fix author
Co-authors
- Jiaheng Liu 3
- Yanghai Wang 3
- Zhe Cao 2
- Zixuan Dong 2
- Shihao Li 2
- Jiafu Tang 2
- Zhiqi Bai 1
- Hanyan Bian 1
- Junlan Feng 1
- Yubin Guo 1
- Xueming Han 1
- Chengkang Jiang 1
- Zhide Lei 1
- Weiliang Luo 1
- Jiayang Mao 1
- Fanyu Meng 1
- Zhiyu Pan 1
- Baoyun Peng 1
- Yuxiang Ren 1
- Jiahao Wang 1
- Jiakai Wang 1
- Xiaodong Wang 1
- Zhaohui Wang 1
- Yuqing Wen 1
- Tiantian Xia 1
- Qianqian Xie 1
- Qingheng Xiong 1
- Yuanxing Zhang 1
- Zhaoxiang Zhang 1
- He Zhu 1