Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss

Wei He, Marco Idiart, Carolina Scarton, Aline Villavicencio


Abstract
Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP). This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics.
Anthology ID:
2024.findings-acl.741
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12473–12485
Language:
URL:
https://aclanthology.org/2024.findings-acl.741
DOI:
Bibkey:
Cite (ACL):
Wei He, Marco Idiart, Carolina Scarton, and Aline Villavicencio. 2024. Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss. In Findings of the Association for Computational Linguistics ACL 2024, pages 12473–12485, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss (He et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.741.pdf