Multi-Level Attention Aggregation for Language-Agnostic Speaker Replication

Yejin Jeon, Gary Lee


Abstract
This paper explores the task of language-agnostic speaker replication, a novel endeavor that seeks to replicate a speaker’s voice irrespective of the language they are speaking. Towards this end, we introduce a multi-level attention aggregation approach that systematically probes and amplifies various speaker-specific attributes in a hierarchical manner. Through rigorous evaluations across a wide range of scenarios including seen and unseen speakers conversing in seen and unseen lingua, we establish that our proposed model is able to achieve substantial speaker similarity, and is able to generalize to out-of-domain (OOD) cases.
Anthology ID:
2024.eacl-short.3
Original:
2024.eacl-short.3v1
Version 2:
2024.eacl-short.3v2
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14–20
Language:
URL:
https://aclanthology.org/2024.eacl-short.3
DOI:
Bibkey:
Cite (ACL):
Yejin Jeon and Gary Lee. 2024. Multi-Level Attention Aggregation for Language-Agnostic Speaker Replication. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 14–20, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Multi-Level Attention Aggregation for Language-Agnostic Speaker Replication (Jeon & Lee, EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-short.3.pdf
Software:
 2024.eacl-short.3.software.zip