Weisheng Xu

Author directory

2026

Recent Large Audio Language Models (LALMs) demonstrate impressive performance on speech benchmarks, yet it remains unclear whether these models genuinely process acoustic signals or rely on text-based semantic inference. To systematically study this question, we introduce DEAF (Diagnostic Evaluation of Acoustic Faithfulness), a bench- mark of over 2,700 conflict stimuli spanning three acoustic dimensions: emotional prosody, background sounds, and speaker identity. Then, we design a controlled multi-level evaluation framework that progressively increases textual influence, ranging from semantic conflicts in the content to misleading prompts and their combination, allowing us to disentangle content-driven bias from prompt-induced sycophancy. We further introduce diagnostic metrics to quantify model reliance on textual cues over acoustic signals. Our evaluation of seven LALMs reveals a consistent pattern of text dominance: models are sensitive to acoustic variations, yet predictions are predominantly driven by textual inputs, revealing a gap between high performance on standard speech benchmarks and genuine acoustic understanding. Code and benchmark resources are publicly available at: https://github.com/kikixiong/DEAF
As text-to-motion (T2M) models move beyond short action captions toward real-world applications, they are increasingly expected to follow diverse instructions. However, existing evaluation protocols remain insufficient for systematically assessing instruction following, physical plausibility, and fine-grained controllability under such diverse language instructions. To address this gap, we introduce T2MBench, a benchmark with 1,025 prompts across four categories—Dynamics, Complexity, Interaction, and Accuracy—covering 14 representative T2M models and three evaluation dimensions: LLM-Based Evaluation, Multi-Factor Motion Evaluation, and Fine-grained Accuracy Evaluation. Our results reveal substantial gaps in current T2M models’ ability to consistently follow diverse language instructions while maintaining motion plausibility and fine-grained control.T2MBench establishes a reusable diagnostic benchmark for advancing research toward more reliable T2M generation under diverse language instructions. Data and materials are available at https://github.com/Courteous121/T2MBench-release.