Syntagmatic Word Embeddings for Unsupervised Learning of Selectional Preferences

Renjith P. Ravindran, Akshay Badola, Narayana Kavi Murthy


Abstract
Selectional Preference (SP) captures the tendency of a word to semantically select other words to be in direct syntactic relation with it, and thus informs us about syntactic word configurations that are meaningful. Therefore SP is a valuable resource for Natural Language Processing (NLP) systems and for semanticists. Learning SP has generally been seen as a supervised task, because it requires a parsed corpus as a source of syntactically related word pairs. In this paper we show that simple distributional analysis can learn a good amount of SP without the need for an annotated corpus. We extend the general word embedding technique with directional word context windows giving word representations that better capture syntagmatic relations. We test on the SP-10K dataset and demonstrate that syntagmatic embeddings outperform the paradigmatic embeddings. We also evaluate supervised version of these embeddings and show that unsupervised syntagmatic embeddings can be as good as supervised embeddings. We also make available the source code of our implementation.
Anthology ID:
2021.repl4nlp-1.22
Volume:
Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Anna Rogers, Iacer Calixto, Ivan Vulić, Naomi Saphra, Nora Kassner, Oana-Maria Camburu, Trapit Bansal, Vered Shwartz
Venue:
RepL4NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
213–222
Language:
URL:
https://aclanthology.org/2021.repl4nlp-1.22
DOI:
10.18653/v1/2021.repl4nlp-1.22
Bibkey:
Cite (ACL):
Renjith P. Ravindran, Akshay Badola, and Narayana Kavi Murthy. 2021. Syntagmatic Word Embeddings for Unsupervised Learning of Selectional Preferences. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 213–222, Online. Association for Computational Linguistics.
Cite (Informal):
Syntagmatic Word Embeddings for Unsupervised Learning of Selectional Preferences (P. Ravindran et al., RepL4NLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.repl4nlp-1.22.pdf
Code
 renjithravindran/spvec
Data
SP-10K