Estimating word co-occurrence probabilities from pretrained static embeddings using a log-bilinear model
Richard
Futrell
author
2022-05
text
Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
Emmanuele
Chersoni
editor
Nora
Hollenstein
editor
Cassandra
Jacobs
editor
Yohei
Oseki
editor
Laurent
Prévot
editor
Enrico
Santus
editor
Association for Computational Linguistics
Dublin, Ireland
conference publication
We investigate how to use pretrained static word embeddings to deliver improved estimates of bilexical co-occurrence probabilities: conditional probabilities of one word given a single other word in a specific relationship. Such probabilities play important roles in psycholinguistics, corpus linguistics, and usage-based cognitive modeling of language more generally. We propose a log-bilinear model taking pretrained vector representations of the two words as input, enabling generalization based on the distributional information contained in both vectors. We show that this model outperforms baselines in estimating probabilities of adjectives given nouns that they attributively modify, and probabilities of nominal direct objects given their head verbs, given limited training data in Arabic, English, Korean, and Spanish.
futrell-2022-estimating
10.18653/v1/2022.cmcl-1.6
https://aclanthology.org/2022.cmcl-1.6
2022-05
54
60