Automatic Learning of Morphological Variations for Handling Out-of-Vocabulary Terms in Urdu-English MT

Nizar Habash, Hayden Metsky


Abstract
We present an approach for online handling of Out-of-Vocabulary (OOV) terms in Urdu-English MT. Since Urdu is morphologically richer than English, we expect a large portion of the OOV terms to be Urdu morphological variations that are irrelevant to English. We describe an approach to automatically learn English-irrelevant (target-irrelevant) Urdu (source) morphological variation rules from standard phrase tables. These rules are learned in an unsupervised (or lightly supervised) manner by exploiting redundancy in Urdu and collocation with English translations. We use these rules to hypothesize in-vocabulary alternatives to the OOV terms. Our results show that we reduce the OOV rate from a standard baseline average of 2.6% to an average of 0.3% (or 89% relative decrease). We also increase the BLEU score by 0.45 (absolute) and 2.8% (relative) on a standard test set. A manual error analysis shows that 28% of handled OOV cases produce acceptable translations in context.
Anthology ID:
2008.amta-papers.9
Volume:
Proceedings of the 8th Conference of the Association for Machine Translation in the Americas: Research Papers
Month:
October 21-25
Year:
2008
Address:
Waikiki, USA
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
107–116
Language:
URL:
https://aclanthology.org/2008.amta-papers.9
DOI:
Bibkey:
Cite (ACL):
Nizar Habash and Hayden Metsky. 2008. Automatic Learning of Morphological Variations for Handling Out-of-Vocabulary Terms in Urdu-English MT. In Proceedings of the 8th Conference of the Association for Machine Translation in the Americas: Research Papers, pages 107–116, Waikiki, USA. Association for Machine Translation in the Americas.
Cite (Informal):
Automatic Learning of Morphological Variations for Handling Out-of-Vocabulary Terms in Urdu-English MT (Habash & Metsky, AMTA 2008)
Copy Citation:
PDF:
https://aclanthology.org/2008.amta-papers.9.pdf