Closing Brackets with Recurrent Neural Networks
Natalia
Skachkova
author
Thomas
Trost
author
Dietrich
Klakow
author
2018-11
text
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Tal
Linzen
editor
Grzegorz
Chrupała
editor
Afra
Alishahi
editor
Association for Computational Linguistics
Brussels, Belgium
conference publication
Many natural and formal languages contain words or symbols that require a matching counterpart for making an expression well-formed. The combination of opening and closing brackets is a typical example of such a construction. Due to their commonness, the ability to follow such rules is important for language modeling. Currently, recurrent neural networks (RNNs) are extensively used for this task. We investigate whether they are capable of learning the rules of opening and closing brackets by applying them to synthetic Dyck languages that consist of different types of brackets. We provide an analysis of the statistical properties of these languages as a baseline and show strengths and limits of Elman-RNNs, GRUs and LSTMs in experiments on random samples of these languages. In terms of perplexity and prediction accuracy, the RNNs get close to the theoretical baseline in most cases.
skachkova-etal-2018-closing
10.18653/v1/W18-5425
https://aclanthology.org/W18-5425
2018-11
232
239