Training and Prediction Data Discrepancies: Challenges of Text Classification with Noisy, Historical Data

R. Andrew Kreek, Emilia Apostolova


Abstract
Industry datasets used for text classification are rarely created for that purpose. In most cases, the data and target predictions are a by-product of accumulated historical data, typically fraught with noise, present in both the text-based document, as well as in the targeted labels. In this work, we address the question of how well performance metrics computed on noisy, historical data reflect the performance on the intended future machine learning model input. The results demonstrate the utility of dirty training datasets used to build prediction models for cleaner (and different) prediction inputs.
Anthology ID:
W18-6114
Volume:
Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
104–109
Language:
URL:
https://aclanthology.org/W18-6114
DOI:
10.18653/v1/W18-6114
Bibkey:
Cite (ACL):
R. Andrew Kreek and Emilia Apostolova. 2018. Training and Prediction Data Discrepancies: Challenges of Text Classification with Noisy, Historical Data. In Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text, pages 104–109, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Training and Prediction Data Discrepancies: Challenges of Text Classification with Noisy, Historical Data (Kreek & Apostolova, WNUT 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-6114.pdf