A Long Hard Look at MWEs in the Age of Language Models

Vered Shwartz


Abstract
In recent years, language models (LMs) have become almost synonymous with NLP. Pre-trained to “read” a large text corpus, such models are useful as both a representation layer as well as a source of world knowledge. But how well do they represent MWEs? This talk will discuss various problems in representing MWEs, and the extent to which LMs address them: • Do LMs capture the implicit relationship between constituents in compositional MWEs (from baby oil through parsley cake to cheeseburger stabbing)? • Do LMs recognize when words are used nonliterally in non-compositional MWEs (e.g. do they know whether there are fleas in the flea market)? • Do LMs know idioms, and can they infer the meaning of new idioms from the context as humans often do?
Anthology ID:
2021.mwe-1.1
Volume:
Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Paul Cook, Jelena Mitrović, Carla Parra Escartín, Ashwini Vaidya, Petya Osenova, Shiva Taslimipoor, Carlos Ramisch
Venue:
MWE
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1
Language:
URL:
https://aclanthology.org/2021.mwe-1.1
DOI:
10.18653/v1/2021.mwe-1.1
Bibkey:
Cite (ACL):
Vered Shwartz. 2021. A Long Hard Look at MWEs in the Age of Language Models. In Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021), page 1, Online. Association for Computational Linguistics.
Cite (Informal):
A Long Hard Look at MWEs in the Age of Language Models (Shwartz, MWE 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.mwe-1.1.pdf