Self Attended Stack-Pointer Networks for Learning Long Term Dependencies

Salih Tuc, Burcu Can


Abstract
We propose a novel deep neural architecture for dependency parsing, which is built upon a Transformer Encoder (Vaswani et al. 2017) and a Stack Pointer Network (Ma et al. 2018). We first encode each sentence using a Transformer Network and then the dependency graph is generated by a Stack Pointer Network by selecting the head of each word in the sentence through a head selection process. We evaluate our model on Turkish and English treebanks. The results show that our trasformer-based model learns long term dependencies efficiently compared to sequential models such as recurrent neural networks. Our self attended stack pointer network improves UAS score around 6% upon the LSTM based stack pointer (Ma et al. 2018) for Turkish sentences with a length of more than 20 words.
Anthology ID:
2020.icon-main.12
Volume:
Proceedings of the 17th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2020
Address:
Indian Institute of Technology Patna, Patna, India
Editors:
Pushpak Bhattacharyya, Dipti Misra Sharma, Rajeev Sangal
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
90–100
Language:
URL:
https://aclanthology.org/2020.icon-main.12
DOI:
Bibkey:
Cite (ACL):
Salih Tuc and Burcu Can. 2020. Self Attended Stack-Pointer Networks for Learning Long Term Dependencies. In Proceedings of the 17th International Conference on Natural Language Processing (ICON), pages 90–100, Indian Institute of Technology Patna, Patna, India. NLP Association of India (NLPAI).
Cite (Informal):
Self Attended Stack-Pointer Networks for Learning Long Term Dependencies (Tuc & Can, ICON 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.icon-main.12.pdf
Data
Penn Treebank