Learning Disentangled Representations for Natural Language Definitions

Danilo Silva De Carvalho, Giangiacomo Mercatali, Yingji Zhang, André Freitas


Abstract
Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. Currently, most disentanglement methods are unsupervised or rely on synthetic datasets with known generative factors. We argue that recurrent syntactic and semantic regularities in textual data can be used to provide the models with both structural biases and generative factors. We leverage the semantic structures present in a representative and semantically dense category of sentence types, definitional sentences, for training a Variational Autoencoder to learn disentangled representations. Our experimental results show that the proposed model outperforms unsupervised baselines on several qualitative and quantitative benchmarks for disentanglement, and it also improves the results in the downstream task of definition modeling.
Anthology ID:
2023.findings-eacl.101
Volume:
Findings of the Association for Computational Linguistics: EACL 2023
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1371–1384
Language:
URL:
https://aclanthology.org/2023.findings-eacl.101
DOI:
10.18653/v1/2023.findings-eacl.101
Bibkey:
Cite (ACL):
Danilo Silva De Carvalho, Giangiacomo Mercatali, Yingji Zhang, and André Freitas. 2023. Learning Disentangled Representations for Natural Language Definitions. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1371–1384, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Learning Disentangled Representations for Natural Language Definitions (Silva De Carvalho et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-eacl.101.pdf
Video:
 https://aclanthology.org/2023.findings-eacl.101.mp4