Convolutions Are All You Need (For Classifying Character Sequences)

Zach Wood-Doughty, Nicholas Andrews, Mark Dredze


Abstract
While recurrent neural networks (RNNs) are widely used for text classification, they demonstrate poor performance and slow convergence when trained on long sequences. When text is modeled as characters instead of words, the longer sequences make RNNs a poor choice. Convolutional neural networks (CNNs), although somewhat less ubiquitous than RNNs, have an internal structure more appropriate for long-distance character dependencies. To better understand how CNNs and RNNs differ in handling long sequences, we use them for text classification tasks in several character-level social media datasets. The CNN models vastly outperform the RNN models in our experiments, suggesting that CNNs are superior to RNNs at learning to classify character-level data.
Anthology ID:
W18-6127
Volume:
Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
208–213
Language:
URL:
https://aclanthology.org/W18-6127
DOI:
10.18653/v1/W18-6127
Bibkey:
Cite (ACL):
Zach Wood-Doughty, Nicholas Andrews, and Mark Dredze. 2018. Convolutions Are All You Need (For Classifying Character Sequences). In Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text, pages 208–213, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Convolutions Are All You Need (For Classifying Character Sequences) (Wood-Doughty et al., WNUT 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-6127.pdf
Data
SST