O_O-VC: Synthetic Data-Driven One-to-One Alignment for Any-to-Any Voice Conversion

Huu Tuong Tu, Huan Vu, Cuong Tien Nguyen, Dien Hy Ngo, Nguyen Thi Thu Trang


Abstract
Traditional voice conversion (VC) methods typically attempt to separate speaker identity and linguistic information into distinct representations, which are then combined to reconstruct the audio. However, effectively disentangling these factors remains challenging, often leading to information loss during training. In this paper, we propose a new approach that leverages synthetic speech data generated by a high-quality, pretrained multispeaker text-to-speech (TTS) model. Specifically, synthetic data pairs that share the same linguistic content but differ in speaker identity are used as input-output pairs to train the voice conversion model. This enables the model to learn a direct mapping between source and target voices, effectively capturing speaker-specific characteristics while preserving linguistic content. Additionally, we introduce a flexible training strategy for any-to-any voice conversion that generalizes well to unseen speakers and new languages, enhancing adaptability and performance in zero-shot scenarios. Our experiments show that our proposed method achieves a 16.35% relative reduction in word error rate and a 5.91% improvement in speaker cosine similarity, outperforming several state-of-the-art methods. Voice conversion samples can be accessed at: https://oovc-emnlp-2025.github.io/
Anthology ID:
2025.findings-emnlp.879
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16197–16208
Language:
URL:
https://aclanthology.org/2025.findings-emnlp.879/
DOI:
Bibkey:
Cite (ACL):
Huu Tuong Tu, Huan Vu, Cuong Tien Nguyen, Dien Hy Ngo, and Nguyen Thi Thu Trang. 2025. O_O-VC: Synthetic Data-Driven One-to-One Alignment for Any-to-Any Voice Conversion. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 16197–16208, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
O_O-VC: Synthetic Data-Driven One-to-One Alignment for Any-to-Any Voice Conversion (Tu et al., Findings 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.findings-emnlp.879.pdf
Checklist:
 2025.findings-emnlp.879.checklist.pdf