Haoxiao Wang
Author directory2026
MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation
Haiyang Sun | Haoxiao Wang | Junming Chen | Weicheng Fang | Zihao Su | Jingkun Yi | Wenyou Yi | Hao Chen | Zhou Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Haiyang Sun | Haoxiao Wang | Junming Chen | Weicheng Fang | Zihao Su | Jingkun Yi | Wenyou Yi | Hao Chen | Zhou Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Robotic manipulation often requires actingon information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the current observation largely determines the next action. Existing robotic memory benchmarks expose this gap, but they still rely mainly on final task success and there-fore conflate forgetting with manipulation failure. We present MEMOBench, a benchmark for process level memory evaluation in robotic manipulation. MEMOBench includes 30 history dependent tasks, 1,500 expert demonstrations, and 4,200 executable checkpoint instances from 84 templates. Each checkpointpairs coarse to fine language with a simulator predicate and labels one memory operation:Storage, Update, or Compression. These annotations define Memory Storage Rate, MemoryUpdate Rate, and Memory Compression Rate,which measure memory fidelity alongside tasksuccess. Across standard and memory augmented VLA policies, the strongest memory module baseline reaches only 31.9%o average success rate, and high storage often coexists with weak update and compression. Checkpoint language also supervises semantic, contrastive, and framewise memory alignment objectives, yielding modest gains across differentmemory operations. MEMOBench provides a diagnostic evaluation suite and training supervision for memory grounded robotic policies. The project page is available at https://github.com/Collab-Gen/MEMoBench.
WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training
Yifu Chen | Shengpeng Ji | Qian Chen | Tianle Liang | Yangzhuo Li | Ziqing Wang | Wen Wang | Jingyu Lu | Haoxiao Wang | Xueyi Pu | Fan Zhuo | Zhou Zhao
Findings of the Association for Computational Linguistics: ACL 2026
Yifu Chen | Shengpeng Ji | Qian Chen | Tianle Liang | Yangzhuo Li | Ziqing Wang | Wen Wang | Jingyu Lu | Haoxiao Wang | Xueyi Pu | Fan Zhuo | Zhou Zhao
Findings of the Association for Computational Linguistics: ACL 2026
End-to-end spoken dialogue models have garnered significant attention because they offer a higher potential ceiling in expressiveness and perceptual ability than cascaded systems. However, the intelligence and expressiveness of current open-source spoken dialogue models often remain below expectations. Motivated by the success of online reinforcement learning(RL) in other domains, one might attempt to directly apply preference optimization to spoken dialogue models, yet this transfer is non-trivial. We analyze these obstacles from the perspectives of reward modeling and rollout sampling, focusing on how sparse preference supervision interacts with dense speech generation under shared-parameter updates. Based on the analysis, we propose a modality-aware adaptive post-training recipe that makes RL practical for spoken dialogue: it constrains preference updates to the semantic channel and improves acoustic behavior via explicit anchoring, while dynamically regulating their mixture from rollout statistics to avoid unreliable preference gradients. We evaluate the method across multiple spoken dialogue benchmarks and representative architectures, and observe consistent improvements in semantic quality and speech expressiveness.
Speaking While Listening: A Survey and Empirical Audit of Full-Duplex Spoken Dialogue Systems
Jingyu Lu | Yuhan Wang | Jianming Luo | Yifu Chen | Tianle Liang | Shengpeng Ji | Ziyue Jiang | Xiaoda Yang | Yu Zhang | Xize Cheng | Chenyuhao Wen | Changhao Pan | Haoxiao Wang | Chen Ye | Jian Wu | Xiaoxi Jiang | Guanjun Jiang | Zhou Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jingyu Lu | Yuhan Wang | Jianming Luo | Yifu Chen | Tianle Liang | Shengpeng Ji | Ziyue Jiang | Xiaoda Yang | Yu Zhang | Xize Cheng | Chenyuhao Wen | Changhao Pan | Haoxiao Wang | Chen Ye | Jian Wu | Xiaoxi Jiang | Guanjun Jiang | Zhou Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
More than a dozen spoken dialogue systems have recently claimed to be "full-duplex," yet the term has been used to describe substantially different capabilities. Existing surveys collapse them onto a single axis (cascaded/end-to-end, or engineered/learned) and miss the distinctions that matter most for builders. We argue that much of this ambiguity is taxonomical: current terminology does not specify where duplex decisions are made, which interaction types are supported, or how a system behaves moment by moment. This paper introduces three complementary frameworks: (i) an L,0-L,3 Architectural Hierarchy that locates where duplex decisions are made; (ii) a T x Ix R Interaction Ontology that specifies the temporal relation, user intent, and required system response for each interaction; and (i) a Decision State Machine(LDLE/LISTEN/SPEAK/WAIT/DUAL) that describes how systems move between states.Across published systems and benchmarks, our audit reveals a realization gap: although many architectures can in principle operate in full-duplex states, their observed behavior remains constrained by the interaction patterns represented in training and evaluation.We identify the open-source-versus-industrial data gap, as well as the still-unrealized goal of L3 representation-level modeling, as key frontiers for future research on full-duplex dialogue. The related material is available at https://github.com/MM-Speech/DuplexSurvey.
2025
WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models
Yifu Chen | Shengpeng Ji | Haoxiao Wang | Ziqing Wang | Siyu Chen | Jinzheng He | Jin Xu | Zhou Zhao
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yifu Chen | Shengpeng Ji | Haoxiao Wang | Ziqing Wang | Siyu Chen | Jinzheng He | Jin Xu | Zhou Zhao
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Retrieval Augmented Generation (RAG) has gained widespread adoption owing to its capacity to empower large language models (LLMs) to integrate external knowledge. However, existing RAG frameworks are primarily designed for text-based LLMs and rely on Automatic Speech Recognition to process speech input, which discards crucial audio information, risks transcription errors, and increases computational overhead. Therefore, we introduce WavRAG, the first retrieval augmented generation framework with native, end-to-end audio support. WavRAG offers two key features: 1) Bypassing ASR, WavRAG directly processes raw audio for both embedding and retrieval. 2) WavRAG integrates audio and text into a unified knowledge representation. Specifically, we propose the WavRetriever to facilitate the retrieval from a text-audio hybrid knowledge base, and further enhance the in-context capabilities of spoken dialogue models through the integration of chain-of-thought reasoning. In comparison to state-of-the-art ASR-Text RAG pipelines, WavRAG achieves comparable retrieval performance while delivering a 10x acceleration. Furthermore, WavRAG’s unique text-audio hybrid retrieval capability extends the boundaries of RAG to the audio modality.
Search
Fix author
Co-authors
- Zhou Zhao 4
- Yifu Chen 3
- Shengpeng Ji 3
- Tianle Liang 2
- Jingyu Lu 2
- Ziqing Wang 2
- Hao Chen 1
- Junming Chen 1
- Qian Chen 1
- Siyu Chen 1
- Xize Cheng 1
- Weicheng Fang 1
- Jinzheng He 1
- Guanjun Jiang 1
- Xiaoxi Jiang 1
- Ziyue Jiang 1
- Yangzhuo Li 1
- Jianming Luo 1
- Changhao Pan 1
- Xueyi Pu 1
- Zihao Su 1
- Haiyang Sun 1
- Wen Wang (王雯) 1
- Yuhan Wang 1
- Chenyuhao Wen 1
- Jian Wu 1
- Jin Xu 1
- Xiaoda Yang 1
- Chen Ye 1
- Jingkun Yi 1
- Wenyou Yi 1
- Yu Zhang 1
- Fan Zhuo 1