An Attention Ensemble Approach for Efficient Text Classification of Indian Languages

Atharva Kulkarni, Amey Hengle, Rutuja Udyawar


Abstract
The recent surge of complex attention-based deep learning architectures has led to extraordinary results in various downstream NLP tasks in the English language. However, such research for resource-constrained and morphologically rich Indian vernacular languages has been relatively limited. This paper proffers a solution for the TechDOfication 2020 subtask-1f: which focuses on the coarse-grained technical domain identification of short text documents in Marathi, a Devanagari script-based Indian language. Availing the large dataset at hand, a hybrid CNN-BiLSTM attention ensemble model is proposed that competently combines the intermediate sentence representations generated by the convolutional neural network and the bidirectional long short-term memory, leading to efficient text classification. Experimental results show that the proposed model outperforms various baseline machine learning and deep learning models in the given task, giving the best validation accuracy of 89.57% and f1-score of 0.8875. Furthermore, the solution resulted in the best system submission for this subtask, giving a test accuracy of 64.26% and f1-score of 0.6157, transcending the performances of other teams as well as the baseline system given by the organizers of the shared task.
Anthology ID:
2020.icon-techdofication.9
Volume:
Proceedings of the 17th International Conference on Natural Language Processing (ICON): TechDOfication 2020 Shared Task
Month:
December
Year:
2020
Address:
Patna, India
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
40–46
Language:
URL:
https://aclanthology.org/2020.icon-techdofication.9
DOI:
Bibkey:
Cite (ACL):
Atharva Kulkarni, Amey Hengle, and Rutuja Udyawar. 2020. An Attention Ensemble Approach for Efficient Text Classification of Indian Languages. In Proceedings of the 17th International Conference on Natural Language Processing (ICON): TechDOfication 2020 Shared Task, pages 40–46, Patna, India. NLP Association of India (NLPAI).
Cite (Informal):
An Attention Ensemble Approach for Efficient Text Classification of Indian Languages (Kulkarni et al., ICON 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.icon-techdofication.9.pdf