Weisheng Xu
Author directory2026
DEAF: A Benchmark for Diagnostic Evaluation of Acoustic Faithfulness in Large Audio Language Models
Jiaqi Xiong | Yunjia Qi | Qi Cao | Yu Zheng | Yutong Zhang | Ziteng Wang | Ruofan Liao | Weisheng Xu | Sichen Liu
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Jiaqi Xiong | Yunjia Qi | Qi Cao | Yu Zheng | Yutong Zhang | Ziteng Wang | Ruofan Liao | Weisheng Xu | Sichen Liu
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Recent Large Audio Language Models (LALMs) demonstrate impressive performance on speech benchmarks, yet it remains unclear whether these models genuinely process acoustic signals or rely on text-based semantic inference. To systematically study this question, we introduce DEAF (Diagnostic Evaluation of Acoustic Faithfulness), a bench- mark of over 2,700 conflict stimuli spanning three acoustic dimensions: emotional prosody, background sounds, and speaker identity. Then, we design a controlled multi-level evaluation framework that progressively increases textual influence, ranging from semantic conflicts in the content to misleading prompts and their combination, allowing us to disentangle content-driven bias from prompt-induced sycophancy. We further introduce diagnostic metrics to quantify model reliance on textual cues over acoustic signals. Our evaluation of seven LALMs reveals a consistent pattern of text dominance: models are sensitive to acoustic variations, yet predictions are predominantly driven by textual inputs, revealing a gap between high performance on standard speech benchmarks and genuine acoustic understanding. Code and benchmark resources are publicly available at: https://github.com/kikixiong/DEAF
T2MBench: Evaluating Text-to-Motion Generation under Diverse Language Instructions
Bin Yang | Rong OU | Weisheng Xu | Jiaqi Xiong | Xintao Li | Taowen Wang | Luyu Zhu | Yuetong Fang | Xu Jiang | Jing Tan | Renjing Xu
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Bin Yang | Rong OU | Weisheng Xu | Jiaqi Xiong | Xintao Li | Taowen Wang | Luyu Zhu | Yuetong Fang | Xu Jiang | Jing Tan | Renjing Xu
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
As text-to-motion (T2M) models move beyond short action captions toward real-world applications, they are increasingly expected to follow diverse instructions. However, existing evaluation protocols remain insufficient for systematically assessing instruction following, physical plausibility, and fine-grained controllability under such diverse language instructions. To address this gap, we introduce T2MBench, a benchmark with 1,025 prompts across four categories—Dynamics, Complexity, Interaction, and Accuracy—covering 14 representative T2M models and three evaluation dimensions: LLM-Based Evaluation, Multi-Factor Motion Evaluation, and Fine-grained Accuracy Evaluation. Our results reveal substantial gaps in current T2M models’ ability to consistently follow diverse language instructions while maintaining motion plausibility and fine-grained control.T2MBench establishes a reusable diagnostic benchmark for advancing research toward more reliable T2M generation under diverse language instructions. Data and materials are available at https://github.com/Courteous121/T2MBench-release.