Grammar Induction with Neural Language Models: An Unusual Replication

Phu Mon Htut, Kyunghyun Cho, Samuel Bowman


Abstract
Grammar induction is the task of learning syntactic structure without the expert-labeled treebanks (Charniak and Carroll, 1992; Klein and Manning, 2002). Recent work on latent tree learning offers a new family of approaches to this problem by inducing syntactic structure using the supervision from a downstream NLP task (Yogatama et al., 2017; Maillard et al., 2017; Choi et al., 2018). In a recent paper published at ICLR, Shen et al. (2018) introduce such a model and report near state-of-the-art results on the target task of language modeling, and the first strong latent tree learning result on constituency parsing. During the analysis of this model, we discover issues that make the original results hard to trust, including tuning and even training on what is effectively the test set. Here, we analyze the model under different configurations to understand what it learns and to identify the conditions under which it succeeds. We find that this model represents the first empirical success for neural network latent tree learning, and that neural language modeling warrants further study as a setting for grammar induction.
Anthology ID:
W18-5452
Volume:
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Tal Linzen, Grzegorz Chrupała, Afra Alishahi
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
371–373
Language:
URL:
https://aclanthology.org/W18-5452
DOI:
10.18653/v1/W18-5452
Bibkey:
Cite (ACL):
Phu Mon Htut, Kyunghyun Cho, and Samuel Bowman. 2018. Grammar Induction with Neural Language Models: An Unusual Replication. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 371–373, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Grammar Induction with Neural Language Models: An Unusual Replication (Htut et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-5452.pdf
Code
 nyu-mll/PRPN-Analysis
Data
MultiNLIPenn TreebankSNLI