Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations

Antoine Simoulin, Benoit Crabbé


Abstract
We introduce a novel tree-based model that learns its composition function together with its structure. The architecture produces sentence embeddings by composing words according to an induced syntactic tree. The parsing and the composition functions are explicitly connected and, therefore, learned jointly. As a result, the sentence embedding is computed according to an interpretable linguistic pattern and may be used on any downstream task. We evaluate our encoder on downstream tasks, and we observe that it outperforms tree-based models relying on external parsers. In some configurations, it is even competitive with Bert base model. Our model is capable of supporting multiple parser architectures. We exploit this property to conduct an ablation study by comparing different parser initializations. We explore to which extent the trees produced by our model compare with linguistic structures and how this initialization impacts downstream performances. We empirically observe that downstream supervision troubles producing stable parses and preserving linguistically relevant structures.
Anthology ID:
2022.naacl-srw.33
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop
Month:
July
Year:
2022
Address:
Hybrid: Seattle, Washington + Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
267–276
Language:
URL:
https://aclanthology.org/2022.naacl-srw.33
DOI:
10.18653/v1/2022.naacl-srw.33
Bibkey:
Cite (ACL):
Antoine Simoulin and Benoit Crabbé. 2022. Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pages 267–276, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
Cite (Informal):
Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations (Simoulin & Crabbé, NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-srw.33.pdf
Video:
 https://aclanthology.org/2022.naacl-srw.33.mp4
Data
Penn TreebankSNLI