Cooperative Learning of Disjoint Syntax and Semantics

Serhii Havrylov, Germán Kruszewski, Armand Joulin


Abstract
There has been considerable attention devoted to models that learn to jointly infer an expression’s syntactic structure and its semantics. Yet, Nangia and Bowman (2018) has recently shown that the current best systems fail to learn the correct parsing strategy on mathematical expressions generated from a simple context-free grammar. In this work, we present a recursive model inspired by Choi et al. (2018) that reaches near perfect accuracy on this task. Our model is composed of two separated modules for syntax and semantics. They are cooperatively trained with standard continuous and discrete optimisation schemes. Our model does not require any linguistic structure for supervision, and its recursive nature allows for out-of-domain generalisation. Additionally, our approach performs competitively on several natural language tasks, such as Natural Language Inference and Sentiment Analysis.
Anthology ID:
N19-1115
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1118–1128
Language:
URL:
https://aclanthology.org/N19-1115
DOI:
10.18653/v1/N19-1115
Bibkey:
Cite (ACL):
Serhii Havrylov, Germán Kruszewski, and Armand Joulin. 2019. Cooperative Learning of Disjoint Syntax and Semantics. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1118–1128, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Cooperative Learning of Disjoint Syntax and Semantics (Havrylov et al., NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1115.pdf
Presentation:
 N19-1115.Presentation.pdf
Video:
 https://aclanthology.org/N19-1115.mp4
Code
 facebookresearch/latent-treelstm
Data
ListOpsMultiNLISNLISSTSST-2