A comparative study of transformer and transfer learning MT models for English-Manipuri

Singh Kshetrimayum Boynao, Singh Ningthoujam Avichandra, Meetei Loitongbam Sanayai, Singh Ningthoujam Justwant, Singh Thoudam Doren, Bandyopadhyay Sivaji


Abstract
In this work, we focus on the development of machine translation (MT) models of a lowresource language pair viz. English-Manipuri. Manipuri is one of the eight scheduled languages of the Indian constitution. Manipuri is currently written in two different scripts: one is its original script called Meitei Mayek and the other is the Bengali script. We evaluate the performance of English-Manipuri MT models based on transformer and transfer learning technique. Our MT models are trained using a dataset of 69,065 parallel sentences and validated on 500 sentences. Using 500 test sentences, the English to Manipuri MT models achieved a BLEU score of 19.13 and 29.05 with mT5 and OpenNMT respectively. The results demonstrate that the OpenNMT model significantly outperforms the mT5 model. Additionally, Manipuri to English MT system trained with OpenNMT model reported a BLEU score of 30.90. We also carried out a comparative analysis between the Bengali script and the transliterated Meitei Mayek script for English-Manipuri MT models. This analysis reveals that the transliterated version enhances the MT model performance resulting in a notable +2.35 improvement in the BLEU score.
Anthology ID:
2023.icon-1.81
Volume:
Proceedings of the 20th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2023
Address:
Goa University, Goa, India
Editors:
D. Pawar Jyoti, Lalitha Devi Sobha
Venue:
ICON
SIG:
SIGLEX
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
791–796
Language:
URL:
https://aclanthology.org/2023.icon-1.81
DOI:
Bibkey:
Cite (ACL):
Singh Kshetrimayum Boynao, Singh Ningthoujam Avichandra, Meetei Loitongbam Sanayai, Singh Ningthoujam Justwant, Singh Thoudam Doren, and Bandyopadhyay Sivaji. 2023. A comparative study of transformer and transfer learning MT models for English-Manipuri. In Proceedings of the 20th International Conference on Natural Language Processing (ICON), pages 791–796, Goa University, Goa, India. NLP Association of India (NLPAI).
Cite (Informal):
A comparative study of transformer and transfer learning MT models for English-Manipuri (Kshetrimayum Boynao et al., ICON 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.icon-1.81.pdf