Proactive Hearing Assistants that Isolate Egocentric Conversations

Guilin Hu, Malek Itani, Tuochao Chen, Shyamnath Gollakota


Abstract
We introduce proactive hearing assistants that automatically identify and separate the wearer’s conversation partners, without requiring explicit prompts. Our system operates on egocentric binaural audio and uses the wearer’s self-speech as an anchor, leveraging turn-taking behavior and dialogue dynamics to infer conversational partners and suppress others. To enable real-time, on-device operation, we propose a dual-model architecture: a lightweight streaming model runs every 12.5 ms for low-latency extraction of the conversation partners, while a slower model runs less frequently to capture longer-range conversational dynamics. Results on real-world 2- and 3-speaker conversation test sets, collected with binaural egocentric hardware from 11 participants totaling 6.8 hours, show generalization in identifying and isolating conversational partners in multi-conversation settings. Our work marks a step toward hearing assistants that adapt proactively to conversational dynamics and engagement.
Anthology ID:
2025.emnlp-main.1289
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25377–25394
Language:
URL:
https://aclanthology.org/2025.emnlp-main.1289/
DOI:
Bibkey:
Cite (ACL):
Guilin Hu, Malek Itani, Tuochao Chen, and Shyamnath Gollakota. 2025. Proactive Hearing Assistants that Isolate Egocentric Conversations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25377–25394, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Proactive Hearing Assistants that Isolate Egocentric Conversations (Hu et al., EMNLP 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.emnlp-main.1289.pdf
Checklist:
 2025.emnlp-main.1289.checklist.pdf