Towards Example-Based NMT with Multi-Levenshtein Transformers

Maxime Bouthors, Josep Crego, François Yvon


Abstract
Retrieval-Augmented Machine Translation (RAMT) is attracting growing attention. This is because RAMT not only improves translation metrics, but is also assumed to implement some form of domain adaptation. In this contribution, we study another salient trait of RAMT, its ability to make translation decisions more transparent by allowing users to go back to examples that contributed to these decisions. For this, we propose a novel architecture aiming to increase this transparency. This model adapts a retrieval-augmented version of the Levenshtein Transformer and makes it amenable to simultaneously edit multiple fuzzy matches found in memory. We discuss how to perform training and inference in this model, based on multi-way alignment algorithms and imitation learning. Our experiments show that editing several examples positively impacts translation scores, notably increasing the number of target spans that are copied from existing instances.
Anthology ID:
2023.emnlp-main.113
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1830–1846
Language:
URL:
https://aclanthology.org/2023.emnlp-main.113
DOI:
10.18653/v1/2023.emnlp-main.113
Bibkey:
Cite (ACL):
Maxime Bouthors, Josep Crego, and François Yvon. 2023. Towards Example-Based NMT with Multi-Levenshtein Transformers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1830–1846, Singapore. Association for Computational Linguistics.
Cite (Informal):
Towards Example-Based NMT with Multi-Levenshtein Transformers (Bouthors et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.113.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.113.mp4