EMBRACE: Emotion-Based Representation Across Cultures & Expressions

Pravin Mote, Carlos Busso


Abstract
Building speech emotion recognition (SER) models that generalize across languages remains challenging due to limited annotations, linguistic diversity and heterogeneous label spaces. Current multilingual SER approaches either rely on fully shared models or restrict training to a small set of overlapping emotions that restrict diversity in expressiveness. We propose EMBRACE, a multilingual speech representation for language-agnostic SER. Drawing inspiration from partial-label learning paradigms in computer vision, EMBRACE adapts a unified superset emotion modeling strategy. The approach enables training on mixtures of datasets with non-overlapping emotion categories by selectively masking absent categories during optimization. To balance this complex multi-domain training, the model leverages a knowledge distillation based progressive training procedure that stabilizes multilingual optimization and mitigates catastrophic forgetting. Experiments across seven languages show that EMBRACE improves cross-corpus performance by 31.8% and in-corpus by 15.2% over strong emotional speech representations, confirming its ability to learn discriminative and transferable affective representations.
Anthology ID:
2026.findings-emnlp.2174
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
46509–46520
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2174/
DOI:
Bibkey:
Cite (ACL):
Pravin Mote and Carlos Busso. 2026. EMBRACE: Emotion-Based Representation Across Cultures & Expressions. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 46509–46520, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
EMBRACE: Emotion-Based Representation Across Cultures & Expressions (Mote & Busso, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2174.pdf