Zhiheng Liu
Author directory2026
EvA: An Evidence-First Audio Understanding Paradigm for LALMs
Xinyuan Xie | Shunian Chen | Zhiheng Liu | Yuhao Zhang | Zhiqiang Lv | Liyin Liang | Benyou Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Xinyuan Xie | Shunian Chen | Zhiheng Liu | Yuhao Zhang | Zhiqiang Lv | Liyin Liang | Benyou Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Audio Language Models (LALMs) still struggle in complex acoustic scenes because they often fail to preserve task-relevant acoustic evidence before reasoning begins. We identify this error pattern as the evidence bottleneck: state-of-the-art systems show larger deficits in acoustic evidence extraction than in downstream reasoning, suggesting that upstream perception is often the limiting factor. To address this problem, we propose EvA (Evidence-First Audio), a dual-path architecture that enhances acoustic evidence preservation through hierarchical aggregation and non-compressive, time-aligned fusion. We also build EvA-Perception, a large-scale training set with about 54K event-ordered captions and 500K evidence-grounded QA pairs. Under a unified zero-shot protocol, EvA achieves the best open-source Perception results on MMAU, MMAR, and MMSU, with the largest gains on perception-heavy splits. Human evaluation on open-ended captioning further shows improved fine-grained acoustic coverage and caption quality. These results support the evidence-first hypothesis: stronger audio understanding depends on preserving acoustic evidence before reasoning.
ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine
Junying Chen | Zhenyang Cai | Zhiheng Liu | Yunjin Yang | Rongsheng Wang | Qingying Xiao | Xiangyi Feng | Zhan Su | Jing Guo | Xiang Wan | Guangjun Yu | Haizhou Li | Benyou Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Junying Chen | Zhenyang Cai | Zhiheng Liu | Yunjin Yang | Rongsheng Wang | Qingying Xiao | Xiangyi Feng | Zhan Su | Jing Guo | Xiang Wan | Guangjun Yu | Haizhou Li | Benyou Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Despite the success of large language models (LLMs), their potential in **Traditional Chinese Medicine (TCM)** remains underexplored due to scarce high-quality data and the multimodal nature of TCM diagnostics, which involve looking, listening, smelling, and pulse-taking. These modalities challenge conventional LLMs. To address these challenges, we present **ShizhenGPT**, the first multimodal LLM tailored for TCM. To overcome data scarcity, we curate the largest TCM dataset to date, comprising 100GB+ of text and 200GB+ of multimodal data, including 1.2M images, 200 hours of audio, and physiological signals. ShizhenGPT is pretrained and instruction-tuned to achieve deep TCM knowledge and multimodal reasoning. For evaluation, we collect recent national TCM qualification exams and build a visual benchmark for Medicinal Recognition and Visual Diagnosis. Experiments demonstrate that ShizhenGPT outperforms comparable-scale LLMs and competes with larger proprietary models. Moreover, it leads in TCM visual understanding among existing multimodal LLMs and shows promising results across modalities such as sound, pulse, electronic-nose signals, and vision, paving the way toward more holistic multimodal perception in TCM. We hope that the datasets, models, and methods of this work will inspire further exploration in this field. Code, models, and released data are available at https://github.com/FreedomIntelligence/ShizhenGPT.
2025
Soundwave: Less is More for Speech-Text Alignment in LLMs
Yuhao Zhang | Zhiheng Liu | Fan Bu | Ruiyu Zhang | Benyou Wang | Haizhou Li
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yuhao Zhang | Zhiheng Liu | Fan Bu | Ruiyu Zhang | Benyou Wang | Haizhou Li
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Existing end-to-end speech large language models (LLMs) usually rely on large-scale annotated data for training, while data-efficient training has not been discussed in depth. We focus on two fundamental problems between speech and text: the representation space gap and sequence length inconsistency. We propose Soundwave, which utilizes an efficient training strategy and a novel architecture to address these issues. Results show that Soundwave outperforms other advanced speech LLMs in speech translation and AIR-Bench speech tasks with only a fraction of the training data. Further analysis shows that Soundwave still retains its intelligence during conversation.