Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training

Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, Gholamreza Haffari


Abstract
Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.
Anthology ID:
2022.findings-acl.49
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
582–588
Language:
URL:
https://aclanthology.org/2022.findings-acl.49
DOI:
10.18653/v1/2022.findings-acl.49
Bibkey:
Cite (ACL):
Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, and Gholamreza Haffari. 2022. Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training. In Findings of the Association for Computational Linguistics: ACL 2022, pages 582–588, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training (Vu et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.49.pdf
Video:
 https://aclanthology.org/2022.findings-acl.49.mp4
Code
 trangvu/lodo-nmt