Model-agnostic Methods for Text Classification with Inherent Noise

Kshitij Tayal, Rahul Ghosh, Vipin Kumar


Abstract
Text classification is a fundamental problem, and recently, deep neural networks (DNN) have shown promising results in many natural language tasks. However, their human-level performance relies on high-quality annotations, which are time-consuming and expensive to collect. As we move towards large inexpensive datasets, the inherent label noise degrades the generalization of DNN. While most machine learning literature focuses on building complex networks to handle noise, in this work, we evaluate model-agnostic methods to handle inherent noise in large scale text classification that can be easily incorporated into existing machine learning workflows with minimal interruption. Specifically, we conduct a point-by-point comparative study between several noise-robust methods on three datasets encompassing three popular classification models. To our knowledge, this is the first time such a comprehensive study in text classification encircling popular models and model-agnostic loss methods has been conducted. In this study, we describe our learning and demonstrate the application of our approach, which outperformed baselines by up to 10 % in classification accuracy while requiring no network modifications.
Anthology ID:
2020.coling-industry.19
Volume:
Proceedings of the 28th International Conference on Computational Linguistics: Industry Track
Month:
December
Year:
2020
Address:
Online
Editors:
Ann Clifton, Courtney Napoles
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
202–213
Language:
URL:
https://aclanthology.org/2020.coling-industry.19
DOI:
10.18653/v1/2020.coling-industry.19
Bibkey:
Cite (ACL):
Kshitij Tayal, Rahul Ghosh, and Vipin Kumar. 2020. Model-agnostic Methods for Text Classification with Inherent Noise. In Proceedings of the 28th International Conference on Computational Linguistics: Industry Track, pages 202–213, Online. International Committee on Computational Linguistics.
Cite (Informal):
Model-agnostic Methods for Text Classification with Inherent Noise (Tayal et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-industry.19.pdf