What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris

Mikael Brunila, Jack LaViolette


Abstract
The power of word embeddings is attributed to the linguistic theory that similar words will appear in similar contexts. This idea is specifically invoked by noting that “you shall know a word by the company it keeps,” a quote from British linguist J.R. Firth who, along with his American colleague Zellig Harris, is often credited with the invention of “distributional semantics.” While both Firth and Harris are cited in all major NLP textbooks and many foundational papers, the content and differences between their theories is seldom discussed. Engaging in a close reading of their work, we discover two distinct and in many ways divergent theories of meaning. One focuses exclusively on the internal workings of linguistic forms, while the other invites us to consider words in new company—not just with other linguistic elements, but also in a broader cultural and situational context. Contrasting these theories from the perspective of current debates in NLP, we discover in Firth a figure who could guide the field towards a more culturally grounded notion of semantics. We consider how an expanded notion of “context” might be modeled in practice through two different strategies: comparative stratification and syntagmatic extension.
Anthology ID:
2022.naacl-main.327
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4403–4417
Language:
URL:
https://aclanthology.org/2022.naacl-main.327
DOI:
10.18653/v1/2022.naacl-main.327
Bibkey:
Cite (ACL):
Mikael Brunila and Jack LaViolette. 2022. What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4403–4417, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris (Brunila & LaViolette, NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.327.pdf