Implicitly-Defined Neural Networks for Sequence Labeling

Michaeel Kazi, Brian Thompson


Abstract
In this work, we propose a novel, implicitly-defined neural network architecture and describe a method to compute its components. The proposed architecture forgoes the causality assumption used to formulate recurrent neural networks and instead couples the hidden states of the network, allowing improvement on problems with complex, long-distance dependencies. Initial experiments demonstrate the new architecture outperforms both the Stanford Parser and baseline bidirectional networks on the Penn Treebank Part-of-Speech tagging task and a baseline bidirectional network on an additional artificial random biased walk task.
Anthology ID:
P17-2027
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
172–177
Language:
URL:
https://aclanthology.org/P17-2027
DOI:
10.18653/v1/P17-2027
Bibkey:
Cite (ACL):
Michaeel Kazi and Brian Thompson. 2017. Implicitly-Defined Neural Networks for Sequence Labeling. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 172–177, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Implicitly-Defined Neural Networks for Sequence Labeling (Kazi & Thompson, ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-2027.pdf
Video:
 https://vimeo.com/234955651
Data
Penn Treebank