Zhu Runjie
Author directory2026
EmbodiChat: Towards Zero-Latency Text-driven Talking Face Generation for Embodied Dialogue
Tiecheng Zhang | Huan Yu | Zhu Runjie | Zequan Xu | Wei Wang | Kaixiang Huang | Jin Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tiecheng Zhang | Huan Yu | Zhu Runjie | Zequan Xu | Wei Wang | Kaixiang Huang | Jin Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Seamless translation of natural language into embodied behaviors remains critical for open domain embodied dialogue. We formalize this task as mapping large language model text to synchronized speech and facial animations. Prevailing cascaded Text to Speech (TTS) and Talking Face Generation (TFG) pipelines suffer from prohibitive latency and poor cross platform generalizability. We present EmbodiChat, an end-to-end real-time dialogue system overcoming these bottlenecks. The key insight of EmbodiChat is that TTS acoustic representations capture pure “articulatory intent,” making them far more robust than audio-derived semantic features. Aligning facial generation directly onto this representation without additional adapters enables decoupled audio visual synthesis, fundamentally eliminating the serial latency of conventional audio first architectures. Concurrently, our framework decodes these representations into hardware agnostic action sequences. EmbodiChat achieves a stable Time-to-First-Audio (TTFA) of about 150 ms on a consumer GPU, significantly outperforming mainstream methods. Furthermore, it delivers a 3x speedup relative to its ablated audio driven baseline while maintaining comparable animation quality. Deployment across two distinct embodied agent platforms configurations demonstrates a robust, practically deployable text driven solution for embodied dialogue. The data, code and model will be released at https://github.com/JiangMingZi/Embodi_Chat