A Study of Morphological Robustness of Neural Machine Translation

Sai Muralidhar Jayanthi, Adithya Pratapa


Abstract
In this work, we analyze the robustness of neural machine translation systems towards grammatical perturbations in the source. In particular, we focus on morphological inflection related perturbations. While this has been recently studied for English→French (MORPHEUS) (Tan et al., 2020), it is unclear how this extends to Any→English translation systems. We propose MORPHEUS-MULTILINGUAL that utilizes UniMorph dictionaries to identify morphological perturbations to source that adversely affect the translation models. Along with an analysis of state-of-the-art pretrained MT systems, we train and analyze systems for 11 language pairs using the multilingual TED corpus (Qi et al., 2018). We also compare this to actual errors of non-native speakers using Grammatical Error Correction datasets. Finally, we present a qualitative and quantitative analysis of the robustness of Any→English translation systems.
Anthology ID:
2021.sigmorphon-1.6
Volume:
Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
Month:
August
Year:
2021
Address:
Online
Editors:
Garrett Nicolai, Kyle Gorman, Ryan Cotterell
Venue:
SIGMORPHON
SIG:
SIGMORPHON
Publisher:
Association for Computational Linguistics
Note:
Pages:
49–59
Language:
URL:
https://aclanthology.org/2021.sigmorphon-1.6
DOI:
10.18653/v1/2021.sigmorphon-1.6
Bibkey:
Cite (ACL):
Sai Muralidhar Jayanthi and Adithya Pratapa. 2021. A Study of Morphological Robustness of Neural Machine Translation. In Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 49–59, Online. Association for Computational Linguistics.
Cite (Informal):
A Study of Morphological Robustness of Neural Machine Translation (Jayanthi & Pratapa, SIGMORPHON 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.sigmorphon-1.6.pdf
Video:
 https://aclanthology.org/2021.sigmorphon-1.6.mp4
Code
 murali1996/morpheus_multilingual