hinglishNorm - A Corpus of Hindi-English Code Mixed Sentences for Text Normalization

Piyush Makhija, Ankit Kumar, Anuj Gupta


Abstract
We present hinglishNorm - a human annotated corpus of Hindi-English code-mixed sentences for text normalization task. Each sentence in the corpus is aligned to its corresponding human annotated normalized form. To the best of our knowledge, there is no corpus of Hindi-English code-mixed sentences for text normalization task that is publicly available. Our work is the first attempt in this direction. The corpus contains 13494 segments annotated for text normalization. Further, we present baseline normalization results on this corpus. We obtain a Word Error Rate (WER) of 15.55, BiLingual Evaluation Understudy (BLEU) score of 71.2, and Metric for Evaluation of Translation with Explicit ORdering (METEOR) score of 0.50.
Anthology ID:
2020.coling-industry.13
Volume:
Proceedings of the 28th International Conference on Computational Linguistics: Industry Track
Month:
December
Year:
2020
Address:
Online
Editors:
Ann Clifton, Courtney Napoles
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
136–145
Language:
URL:
https://aclanthology.org/2020.coling-industry.13
DOI:
10.18653/v1/2020.coling-industry.13
Bibkey:
Cite (ACL):
Piyush Makhija, Ankit Kumar, and Anuj Gupta. 2020. hinglishNorm - A Corpus of Hindi-English Code Mixed Sentences for Text Normalization. In Proceedings of the 28th International Conference on Computational Linguistics: Industry Track, pages 136–145, Online. International Committee on Computational Linguistics.
Cite (Informal):
hinglishNorm - A Corpus of Hindi-English Code Mixed Sentences for Text Normalization (Makhija et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-industry.13.pdf
Data
MTNT