Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text

Vivek Srivastava, Mayank Singh


Abstract
In this shared task, we seek the participating teams to investigate the factors influencing the quality of the code-mixed text generation systems. We synthetically generate code-mixed Hinglish sentences using two distinct approaches and employ human annotators to rate the generation quality. We propose two subtasks, quality rating prediction and annotators’ disagreement prediction of the synthetic Hinglish dataset. The proposed subtasks will put forward the reasoning and explanation of the factors influencing the quality and human perception of the code-mixed text.
Anthology ID:
2021.inlg-1.34
Volume:
Proceedings of the 14th International Conference on Natural Language Generation
Month:
August
Year:
2021
Address:
Aberdeen, Scotland, UK
Editors:
Anya Belz, Angela Fan, Ehud Reiter, Yaji Sripada
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
314–319
Language:
URL:
https://aclanthology.org/2021.inlg-1.34
DOI:
10.18653/v1/2021.inlg-1.34
Bibkey:
Cite (ACL):
Vivek Srivastava and Mayank Singh. 2021. Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text. In Proceedings of the 14th International Conference on Natural Language Generation, pages 314–319, Aberdeen, Scotland, UK. Association for Computational Linguistics.
Cite (Informal):
Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text (Srivastava & Singh, INLG 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.inlg-1.34.pdf