Importance-Aware Data Augmentation for Document-Level Neural Machine Translation

Minghao Wu, Yufei Wang, George Foster, Lizhen Qu, Gholamreza Haffari


Abstract
Document-level neural machine translation (DocNMT) aims to generate translations that are both coherent and cohesive, in contrast to its sentence-level counterpart. However, due to its longer input length and limited availability of training data, DocNMT often faces the challenge of data sparsity. To overcome this issue, we propose a novel Importance-Aware Data Augmentation (IADA) algorithm for DocNMT that augments the training data based on token importance information estimated by the norm of hidden states and training gradients. We conduct comprehensive experiments on three widely-used DocNMT benchmarks. Our empirical results show that our proposed IADA outperforms strong DocNMT baselines as well as several data augmentation approaches, with statistical significance on both sentence-level and document-level BLEU.
Anthology ID:
2024.eacl-long.44
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
740–752
Language:
URL:
https://aclanthology.org/2024.eacl-long.44
DOI:
Bibkey:
Cite (ACL):
Minghao Wu, Yufei Wang, George Foster, Lizhen Qu, and Gholamreza Haffari. 2024. Importance-Aware Data Augmentation for Document-Level Neural Machine Translation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 740–752, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Importance-Aware Data Augmentation for Document-Level Neural Machine Translation (Wu et al., EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-long.44.pdf
Video:
 https://aclanthology.org/2024.eacl-long.44.mp4