Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora

Marcin Junczys-Dowmunt


Abstract
In this work we introduce dual conditional cross-entropy filtering for noisy parallel data. For each sentence pair of the noisy parallel corpus we compute cross-entropy scores according to two inverse translation models trained on clean data. We penalize divergent cross-entropies and weigh the penalty by the cross-entropy average of both models. Sorting or thresholding according to these scores results in better subsets of parallel data. We achieve higher BLEU scores with models trained on parallel data filtered only from Paracrawl than with models trained on clean WMT data. We further evaluate our method in the context of the WMT2018 shared task on parallel corpus filtering and achieve the overall highest ranking scores of the shared task, scoring top in three out of four subtasks.
Anthology ID:
W18-6478
Original:
W18-6478v1
Version 2:
W18-6478v2
Volume:
Proceedings of the Third Conference on Machine Translation: Shared Task Papers
Month:
October
Year:
2018
Address:
Belgium, Brussels
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
888–895
Language:
URL:
https://aclanthology.org/W18-6478
DOI:
10.18653/v1/W18-6478
Bibkey:
Cite (ACL):
Marcin Junczys-Dowmunt. 2018. Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 888–895, Belgium, Brussels. Association for Computational Linguistics.
Cite (Informal):
Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora (Junczys-Dowmunt, WMT 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-6478.pdf