Scrambled Translation Problem: A Problem of Denoising UNMT

Tamali Banerjee, Rudra V Murthy, Pushpak Bhattacharya


Abstract
In this paper and we identify an interesting kind of error in the output of Unsupervised Neural Machine Translation (UNMT) systems like Undreamt1. We refer to this error type as Scrambled Translation problem. We observe that UNMT models which use word shuffle noise (as in case of Undreamt) can generate correct words and but fail to stitch them together to form phrases. As a result and words of the translated sentence look scrambled and resulting in decreased BLEU. We hypothesise that the reason behind scrambled translation problem is ’shuffling noise’ which is introduced in every input sentence as a denoising strategy. To test our hypothesis and we experiment by retraining UNMT models with a simple retraining strategy. We stop the training of the Denoising UNMT model after a pre-decided number of iterations and resume the training for the remaining iterations- which number is also pre-decided- using original sentence as input without adding any noise. Our proposed solution achieves significant performance improvement UNMT models that train conventionally. We demonstrate these performance gains on four language pairs and viz. and English-French and English-German and English-Spanish and Hindi-Punjabi. Our qualitative and quantitative analysis shows that the retraining strategy helps achieve better alignment as observed by attention heatmap and better phrasal translation and leading to statistically significant improvement in BLEU scores.
Anthology ID:
2021.mtsummit-research.11
Volume:
Proceedings of Machine Translation Summit XVIII: Research Track
Month:
August
Year:
2021
Address:
Virtual
Editors:
Kevin Duh, Francisco Guzmán
Venue:
MTSummit
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
127–138
Language:
URL:
https://aclanthology.org/2021.mtsummit-research.11
DOI:
Bibkey:
Cite (ACL):
Tamali Banerjee, Rudra V Murthy, and Pushpak Bhattacharya. 2021. Scrambled Translation Problem: A Problem of Denoising UNMT. In Proceedings of Machine Translation Summit XVIII: Research Track, pages 127–138, Virtual. Association for Machine Translation in the Americas.
Cite (Informal):
Scrambled Translation Problem: A Problem of Denoising UNMT (Banerjee et al., MTSummit 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.mtsummit-research.11.pdf