Learning Context-free Languages with Nondeterministic Stack RNNs

Brian DuSell, David Chiang


Abstract
We present a differentiable stack data structure that simultaneously and tractably encodes an exponential number of stack configurations, based on Lang’s algorithm for simulating nondeterministic pushdown automata. We call the combination of this data structure with a recurrent neural network (RNN) controller a Nondeterministic Stack RNN. We compare our model against existing stack RNNs on various formal languages, demonstrating that our model converges more reliably to algorithmic behavior on deterministic tasks, and achieves lower cross-entropy on inherently nondeterministic tasks.
Anthology ID:
2020.conll-1.41
Volume:
Proceedings of the 24th Conference on Computational Natural Language Learning
Month:
November
Year:
2020
Address:
Online
Editors:
Raquel Fernández, Tal Linzen
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
507–519
Language:
URL:
https://aclanthology.org/2020.conll-1.41
DOI:
10.18653/v1/2020.conll-1.41
Bibkey:
Cite (ACL):
Brian DuSell and David Chiang. 2020. Learning Context-free Languages with Nondeterministic Stack RNNs. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 507–519, Online. Association for Computational Linguistics.
Cite (Informal):
Learning Context-free Languages with Nondeterministic Stack RNNs (DuSell & Chiang, CoNLL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.conll-1.41.pdf
Code
 bdusell/nondeterministic-stack-rnn