Yinghua Qin

Author directory

2026

Conversational emotion recognition suffers from two compounding limitations: models trained on imbalanced data report near-zero F1 on minority emotions, and evaluation has been almost exclusively English, leaving it unknown whether current methods generalize across languages. We propose UniMER, a parameter-efficient framework built on the insight that natural language is the universal interface: visual, acoustic, and facial cues are converted into text by a multimodal LLM, enabling a single LoRA-adapted backbone to process all modalities and languages through one shared representation. To prevent minority-class collapse under extreme imbalance, we introduce class-prototype attention pooling, which anchors each attention head to a learnable class prototype so every emotion retains dedicated representational capacity. With only 2.4% trainable parameters, UniMER achieves state-of-the-art results on three benchmarks across three language families: MELD (English, +14.0pp macro F1), M3ED (Chinese, +3.2pp wF1, text-only), and AkaCE (Akan, +11.5pp macro F1), lifting AkaCE Fear and Disgust F1 from near-zero (<0.1%) to 58% and 40%. All three benchmarks share the 7-class Ekman taxonomy, enabling direct cross-dataset transfer evaluation. Cross-lingual analysis reveals universal arousal-based confusions alongside culturally specific patterns: heightened neutral absorption in Chinese and prosodic confusion in Akan absent in English.