Yupeng Hu
Author directory2026
CoVeR-VQA: Multi-Stage Evidence Verification for GoldenViewVQA
Kun Wang | Yupeng Hu | Ruping Cao | Hao Liu | Zhiran Li | Qianlong Xiang | Harry Cheng
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Kun Wang | Yupeng Hu | Ruping Cao | Hao Liu | Zhiran Li | Qianlong Xiang | Harry Cheng
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
GoldenViewVQA requires models to jointly answer driving-scene questions and identify the camera view containing the supporting visual evidence, making precise evidence localization as important as answer correctness. We present **CoVeR-VQA**, a training-free multi-stage verification and correction framework for grounded multi-view VQA. Starting from GPT-5.6 zero-shot predictions, CoVeR-VQA progressively applies view-specific verification with Gemini-3.6-Flash, prior-guided joint verification with Claude-Opus-5, and cross-split group-level verification that exploits semantically filtered question groups from shared multi-view scenes and validation-derived prior knowledge. On the official GoldenViewVQA test set, the four-stage CoVeR-VQA pipeline achieves 84.75% Joint Accuracy, improving the GPT-5.6 zero-shot baseline by 13.56 percentage points, while reaching 94.92% Answer Accuracy and 86.44% View Accuracy. The final submitted run achieves 88.14% Joint Accuracy after two additional evaluator-informed post-hoc corrections. Our analysis shows that supporting-view localization remains the primary source of residual errors, highlighting the importance of explicit evidence verification for reliable multi-view multimodal reasoning.
Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Space
Chao Chen | Zhixin Ma | Yongqi Li | Yupeng Hu | Yinwei Wei | Wenjie Li | Liqiang Nie
Findings of the Association for Computational Linguistics: ACL 2026
Chao Chen | Zhixin Ma | Yongqi Li | Yupeng Hu | Yinwei Wei | Wenjie Li | Liqiang Nie
Findings of the Association for Computational Linguistics: ACL 2026
Multimodal reasoning aims to enhance the capabilities of MLLMs by incorporating intermediate reasoning steps before reaching the final answer. It has evolved from text-only reasoning to the integration of visual information, enabling the thought process to be conveyed through both images and text. Despite its effectiveness, current multimodal reasoning methods depend on explicit reasoning steps that require labor-intensive vision-text annotations and inherently introduce significant inference latency. To address these issues, we introduce multimodal latent reasoning with the advantages of multimodal representation, reduced annotation, and inference efficiency. To facilicate it, we propose Interleaved Vision-Text Latent Reasoning (IVT-LR), which injects both visual and textual information in the reasoning process within the latent space. Specifically, IVT-LR represents each reasoning step by combining two implicit parts: latent text (the hidden states from the previous step) and latent vision (a set of selected image embeddings). We further introduce a progressive multi-stage training strategy to enable MLLMs to perform the above multimodal latent reasoning steps. Experiments on M3CoT and ScienceQA demonstrate that our IVT-LR method achieves an average performance increase of 5.45% in accuracy, while simultaneously achieving a speed increase of over 5 times compared to existing approaches. The code are attached in the supplementary file for the review.
Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
Dongjie Cheng | Yongqi Li | Zhixin Ma | Hongru Cai | Yupeng Hu | Wenjie Wang | Liqiang Nie | Wenjie Li
Findings of the Association for Computational Linguistics: ACL 2026
Dongjie Cheng | Yongqi Li | Zhixin Ma | Hongru Cai | Yupeng Hu | Wenjie Wang | Liqiang Nie | Wenjie Li
Findings of the Association for Computational Linguistics: ACL 2026
Multimodal Large Language Models (MLLMs) are making significant progress in multimodal reasoning. Early approaches focus on pure text-based reasoning. More recent studies have incorporated multimodal information into the reasoning steps; however, they often follow a single task-specific reasoning pattern, which limits their generalizability across various multimodal tasks. In fact, there are numerous multimodal tasks requiring diverse reasoning skills, such as zooming in on a specific region or marking an object within an image. To address this, we propose unified generative multimodal reasoning, which unifies diverse multimodal reasoning skills by generating intermediate images during the reasoning process. We instantiate this paradigm with Omni-R1, a two-stage SFT+RL framework featuring perception alignment loss and perception reward, thereby enabling functional image generation. Additionally, we introduce Omni-R1-Zero, which eliminates the need for multimodal annotations by bootstrapping step-wise visualizations from text-only reasoning data. Empirical results show that Omni-R1 achieves unified generative reasoning across a wide range of multimodal tasks, and Omni-R1-Zero can match or even surpass Omni-R1 on average, suggesting a promising direction for generative multimodal reasoning. The code and checkpoints are attached for reproducibility and subsequent open release.
RACA-PODS: Reasoning-Informed First-Token Prediction for Query-Document Relevance in WeChat Search
Yupeng Hu | Dongliang Liao | Changxing Ding | Mengchuan Qiu | Huasheng Liang | Qiang Yan | Junhong Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Yupeng Hu | Dongliang Liao | Changxing Ding | Mengchuan Qiu | Huasheng Liang | Qiang Yan | Junhong Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Applying LLMs to query-document relevance estimation in industrial document retrieval faces two key challenges: the complexity and heterogeneity of Web content, and stringent latency and resource constraints. To address them, we propose an automated framework with two components: **R**eflection-**A**ware **C**ollaborative **A**gent (**RACA**) and **P**rogressive **O**ptimization **D**istillation **S**trategy (**PODS**). RACA uses collaborative reasoning and reflection to produce relevance labels and structured rationales. PODS stratifies training data by difficulty and combines supervised fine-tuning with Reasoning-Informed GRPO. Rationale supervision improves the student’s first-token relevance decision, while deployment stops after that token and avoids explicit rationale generation. Together, these mechanisms enable reasoning-informed first-token prediction that preserves the benefits of structured rationale supervision under stringent latency and resource constraints. Extensive offline experiments and a 10-day online A/B test in WeChat Search demonstrate the resulting accuracy–efficiency trade-off.
Resonating with RoPE: Spectral Quantization for High-Fidelity Key Cache Compression
Xuefei Wang | Haoyu Tang | Tianyuan Liang | Zhibin Wang | Yupeng Hu | Weili Guan
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Xuefei Wang | Haoyu Tang | Tianyuan Liang | Zhibin Wang | Yupeng Hu | Weili Guan
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
The linear growth of KV cache bottlenecks long-context LLMs, yet RoPE-induced oscillations complicate Key cache quantization. To address this issue, we propose SpectrumQuant, a frequency-domain framework that utilizes the Discrete Cosine Transform (DCT) to convert these oscillations into sparse spectral representations. Specifically, our pipeline integrates dominant frequency extraction, hybrid bit-width allocation, and high-frequency pre-emphasis to maximize fidelity while minimizing memory footprint. To eliminate computational overhead, we develop fused Triton kernels featuring deferred inverse transformation and on-chip sparse accumulation. Extensive experiments on several benchmarks confirm SpectrumQuant achieves efficient compression with performance and latency comparable to FP16 baselines.
TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval
Zixu Li | Yupeng Hu | Zhiheng Fu | Zhiwei Chen | Yongqi Li | Liqiang Nie
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zixu Li | Yupeng Hu | Zhiheng Fu | Zhiwei Chen | Yongqi Li | Liqiang Nie
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Composed Image Retrieval (CIR) is an important image retrieval paradigm that enables users to retrieve a target image using a multimodal query that consists of a reference image and modification text. Although research on CIR has made significant progress, prevailing setups still rely simple modification texts that typically cover only a limited range of salient changes, which induces two limitations highly relevant to practical applications, namely Insufficient Entity Coverage and Clause-Entity Misalignment. In order to address these issues and bring CIR closer to real-world use, we construct two instruction-rich multi-modification datasets, M-FashionIQ and M-CIRR. In addition, we propose TEMA, the Text-oriented Entity Mapping Architecture, which is the first CIR framework designed for multi-modification while also accommodating simple modifications. Extensive experiments on four benchmark datasets demonstrate that TEMA’s superiority in both original and multi-modification scenarios, while maintaining an optimal balance between retrieval accuracy and computational efficiency. Our codes and constructed multi-modification dataset (M-FashionIQ and M-CIRR) are available at https://github.com/lee-zixu/ACL26-TEMA/
2025
CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG
Yang Tian | Fan Liu | Jingyuan Zhang | Victoria W. | Yupeng Hu | Liqiang Nie
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yang Tian | Fan Liu | Jingyuan Zhang | Victoria W. | Yupeng Hu | Liqiang Nie
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Multimodal Retrieval-Augmented Generation (MMRAG) has been introduced to enhance Multimodal Large Language Models by incorporating externally retrieved multimodal knowledge, but it introduces two challenges: Parametric-Retrieved Knowledge Inconsistency (PRKI), where discrepancies between parametric and retrieved knowledge create uncertainty in determining reliability, and Visual-Textual Knowledge Inconsistency (VTKI), where misalignment between visual and textual sources disrupts entity representation. To address these challenges, we propose Cross-source knowledge Reconciliation for MultiModal RAG (CoRe-MMRAG), a novel end-to-end framework that effectively reconciles inconsistencies across knowledge sources. CoRe-MMRAG follows a four-stage pipeline: it first generates an internal response from parametric knowledge, then selects the most relevant multimodal evidence via joint similarity assessment, generates an external response, and finally integrates both to produce a reliable answer. Additionally, a specialized training paradigm enhances knowledge source discrimination, multimodal integration, and unified answer generation. Experiments on KB-VQA benchmarks show that CoRe-MMRAG achieves substantial improvements over baseline methods, achieving 5.6% and 9.3% performance gains on InfoSeek and Encyclopedic-VQA, respectively. We release code and data at https://github.com/TyangJN/CoRe-MMRAG.
Search
Fix author
Co-authors
- Liqiang Nie 4
- Yongqi Li 3
- Wenjie Li 2
- Zhixin Ma 2
- Hongru Cai 1
- Ruping Cao 1
- Chao Chen 1
- Zhiwei Chen 1
- Dongjie Cheng 1
- Harry Cheng 1
- Changxing Ding 1
- Zhiheng Fu 1
- Weili Guan 1
- Zhiran Li 1
- Zixu Li 1
- Huasheng Liang 1
- Tianyuan Liang 1
- Dongliang Liao 1
- Fan Liu 1
- Hao Liu 1
- Mengchuan Qiu 1
- Haoyu Tang 1
- Yang Tian 1
- Victoria W. 1
- Junhong Wang 1
- Kun Wang 1
- Wenjie Wang 1
- Xuefei Wang 1
- Zhibin Wang 1
- Yinwei Wei 1
- Qianlong Xiang 1
- Rolan Yan 1
- Jingyuan Zhang 1