Self-Governing Neural Networks for On-Device Short Text Classification

Sujith Ravi, Zornitsa Kozareva


Abstract
Deep neural networks reach state-of-the-art performance for wide range of natural language processing, computer vision and speech applications. Yet, one of the biggest challenges is running these complex networks on devices such as mobile phones or smart watches with tiny memory footprint and low computational capacity. We propose on-device Self-Governing Neural Networks (SGNNs), which learn compact projection vectors with local sensitive hashing. The key advantage of SGNNs over existing work is that they surmount the need for pre-trained word embeddings and complex networks with huge parameters. We conduct extensive evaluation on dialog act classification and show significant improvement over state-of-the-art results. Our findings show that SGNNs are effective at capturing low-dimensional semantic text representations, while maintaining high accuracy.
Anthology ID:
D18-1105
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
887–893
Language:
URL:
https://aclanthology.org/D18-1105
DOI:
10.18653/v1/D18-1105
Bibkey:
Cite (ACL):
Sujith Ravi and Zornitsa Kozareva. 2018. Self-Governing Neural Networks for On-Device Short Text Classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 887–893, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Self-Governing Neural Networks for On-Device Short Text Classification (Ravi & Kozareva, EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1105.pdf
Video:
 https://vimeo.com/305197775