Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units

Arturo Argueta, David Chiang


Abstract
Graphics Processing Units (GPUs) are commonly used to train and evaluate neural networks efficiently. While previous work in deep learning has focused on accelerating operations on dense matrices/tensors on GPUs, efforts have concentrated on operations involving sparse data structures. Operations using sparse structures are common in natural language models at the input and output layers, because these models operate on sequences over discrete alphabets. We present two new GPU algorithms: one at the input layer, for multiplying a matrix by a few-hot vector (generalizing the more common operation of multiplication by a one-hot vector) and one at the output layer, for a fused softmax and top-N selection (commonly used in beam search). Our methods achieve speedups over state-of-the-art parallel GPU baselines of up to 7x and 50x, respectively. We also illustrate how our methods scale on different GPU architectures.
Anthology ID:
P19-1626
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6215–6224
Language:
URL:
https://aclanthology.org/P19-1626
DOI:
10.18653/v1/P19-1626
Bibkey:
Cite (ACL):
Arturo Argueta and David Chiang. 2019. Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6215–6224, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units (Argueta & Chiang, ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1626.pdf