Enhanced Training Methods for Multiple Languages

Hai Li, Yang Li


Abstract
Document-grounded dialogue generation based on multilingual is a challenging and realistic task. Unlike previous tasks, it need to tackle with multiple high-resource languages facilitating low-resource languages. This paper summarizes our research based on a three-stage pipeline that includes retrieval, re-rank and generation where each component is individually optimized. In different languages with limited data scenarios, we mainly improve the robustness of the pipeline through data augmentation and embedding perturbation with purpose of improving the performance designing three training methods: cross-language enhancement training, weighted training with neighborhood distribution augmentation, and ensemble adversarial training, all of that can be used as plug and play modules. Through experiments with different settings, it has been shown that our methods can effectively improve the generalization performance of pipeline with score ranking 6th among the public submissions on leaderboards.
Anthology ID:
2023.dialdoc-1.6
Volume:
Proceedings of the Third DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Smaranda Muresan, Vivian Chen, Kennington Casey, Vandyke David, Dethlefs Nina, Inoue Koji, Ekstedt Erik, Ultes Stefan
Venue:
dialdoc
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
52–56
Language:
URL:
https://aclanthology.org/2023.dialdoc-1.6
DOI:
10.18653/v1/2023.dialdoc-1.6
Bibkey:
Cite (ACL):
Hai Li and Yang Li. 2023. Enhanced Training Methods for Multiple Languages. In Proceedings of the Third DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering, pages 52–56, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Enhanced Training Methods for Multiple Languages (Li & Li, dialdoc 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.dialdoc-1.6.pdf