Encoded but Not Express: Probing Subtle Speech-Conditioned Emotion in Large Audio Language Models

Jingxi Liang, Pengze Guo, Zhiwen Xie, Qifeng Wang, Yujia Huo, Derek F. Wong


Abstract
Large Audio Language Models (LALMs) exhibit strong speech understanding ability, yet it remains unclear whether they meaningfully utilize subtle speech-conditioned emotional cues beyond coarse acted emotions. In this work, we investigate whether the evaluated LALMs exhibit measurable sensitivity to fine-grained emotional variation in speech through controlled perturbation experiments, fine-grained emotion elicitation tasks, and representation geometry analysis. Our results show that vocal delivery variations systematically influence both model behavior and hidden representations, indicating that the evaluated LALMs already encode non-trivial speech-conditioned affective information. However, models struggle under emotionally ambiguous settings, particularly for semantically related fine-grained emotions. Representation analysis further suggests that subtle emotional structure is partially preserved despite weak downstream utilization. Overall, our findings suggest that the evaluated LALMs do not completely ignore subtle speech emotion cues, but instead utilize them inconsistently under ambiguous affective conditions
Anthology ID:
2026.aacl-long.262
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5809–5835
Language:
URL:
https://aclanthology.org/2026.aacl-long.262/
DOI:
Bibkey:
Cite (ACL):
Jingxi Liang, Pengze Guo, Zhiwen Xie, Qifeng Wang, Yujia Huo, and Derek F. Wong. 2026. Encoded but Not Express: Probing Subtle Speech-Conditioned Emotion in Large Audio Language Models. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5809–5835, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Encoded but Not Express: Probing Subtle Speech-Conditioned Emotion in Large Audio Language Models (Liang et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.262.pdf