Development of POS tagger for English-Bengali Code-Mixed data

Tathagata Raha, Sainik Mahata, Dipankar Das, Sivaji Bandyopadhyay


Abstract
Code-mixed texts are widespread nowadays due to the advent of social media. Since these texts combine two languages to formulate a sentence, it gives rise to various research problems related to Natural Language Processing. In this paper, we try to excavate one such problem, namely, Parts of Speech tagging of code-mixed texts. We have built a system that can POS tag English-Bengali code-mixed data where the Bengali words were written in Roman script. Our approach initially involves the collection and cleaning of English-Bengali code-mixed tweets. These tweets were used as a development dataset for building our system. The proposed system is a modular approach that starts by tagging individual tokens with their respective languages and then passes them to different POS taggers, designed for different languages (English and Bengali, in our case). Tags given by the two systems are later joined together and the final result is then mapped to a universal POS tag set. Our system was checked using 100 manually POS tagged code-mixed sentences and it returned an accuracy of 75.29%.
Anthology ID:
2019.icon-1.17
Volume:
Proceedings of the 16th International Conference on Natural Language Processing
Month:
December
Year:
2019
Address:
International Institute of Information Technology, Hyderabad, India
Venue:
ICON
SIG:
Publisher:
NLP Association of India
Note:
Pages:
143–149
Language:
URL:
https://aclanthology.org/2019.icon-1.17
DOI:
Bibkey:
Cite (ACL):
Tathagata Raha, Sainik Mahata, Dipankar Das, and Sivaji Bandyopadhyay. 2019. Development of POS tagger for English-Bengali Code-Mixed data. In Proceedings of the 16th International Conference on Natural Language Processing, pages 143–149, International Institute of Information Technology, Hyderabad, India. NLP Association of India.
Cite (Informal):
Development of POS tagger for English-Bengali Code-Mixed data (Raha et al., ICON 2019)
Copy Citation:
PDF:
https://aclanthology.org/2019.icon-1.17.pdf