Do latent tree learning models identify meaningful structure in sentences?

Adina Williams, Andrew Drozdov, Samuel R. Bowman


Abstract
Recent work on the problem of latent tree learning has made it possible to train neural networks that learn to both parse a sentence and use the resulting parse to interpret the sentence, all without exposure to ground-truth parse trees at training time. Surprisingly, these models often perform better at sentence understanding tasks than models that use parse trees from conventional parsers. This paper aims to investigate what these latent tree learning models learn. We replicate two such models in a shared codebase and find that (i) only one of these models outperforms conventional tree-structured models on sentence classification, (ii) its parsing strategies are not especially consistent across random restarts, (iii) the parses it produces tend to be shallower than standard Penn Treebank (PTB) parses, and (iv) they do not resemble those of PTB or any other semantic or syntactic formalism that the authors are aware of.
Anthology ID:
Q18-1019
Volume:
Transactions of the Association for Computational Linguistics, Volume 6
Month:
Year:
2018
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
253–267
Language:
URL:
https://aclanthology.org/Q18-1019
DOI:
10.1162/tacl_a_00019
Bibkey:
Cite (ACL):
Adina Williams, Andrew Drozdov, and Samuel R. Bowman. 2018. Do latent tree learning models identify meaningful structure in sentences?. Transactions of the Association for Computational Linguistics, 6:253–267.
Cite (Informal):
Do latent tree learning models identify meaningful structure in sentences? (Williams et al., TACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/Q18-1019.pdf
Video:
 https://vimeo.com/277673973
Code
 NYU-MLL/spinn
Data
MultiNLISNLI