EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis

Tianchi Liu, Zeyang Song, Tianrui Wang, Zhipeng Li, Chenglin Xu, Steven Y. Guo


Abstract
Psychological research on emotion dynamics has established that human affect is a continuous, evolving process: emotions rise, decay, and transition within seconds. Current emotional text-to-speech (TTS) systems, however, condition on a single discrete label or static embedding per utterance, fundamentally misaligning with the temporal nature of affect. While recent LLM-based TTS systems may implicitly vary prosody through text understanding, such variation is neither explicitly controllable nor precise enough for targeted intra-utterance transitions. We address three challenges: (1) a multi-pass flow blending pipeline synthesizes frame-aligned transition audio, circumventing the scarcity of natural intra-utterance transitions; (2) dual-stage Valence–Arousal–Dominance (VAD) conditioning guides prosodic planning in the LLM and acoustic realization in the flow decoder via frame-level VAD embeddings; (3) direction–magnitude decoupled injection structurally separates emotion direction from injection magnitude, preventing content degradation. EmoTra-TTS adds only +0.43% parameters with no latency overhead, achieves 30%–87% relative improvement on emotion transition quality, corroborated by 67.5%–79.5% overall win rates in a pairwise preference test against four SOTA baselines. The complete code, including data synthesis and training pipelines, will be released after the review process.
Anthology ID:
2026.emnlp-main.736
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
17073–17091
Language:
URL:
https://aclanthology.org/2026.emnlp-main.736/
DOI:
Bibkey:
Cite (ACL):
Tianchi Liu, Zeyang Song, Tianrui Wang, Zhipeng Li, Chenglin Xu, and Steven Y. Guo. 2026. EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 17073–17091, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis (Liu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.736.pdf