JARVix at SemEval-2022 Task 2: It Takes One to Know One? Idiomaticity Detection using Zero and One-Shot Learning

Yash Jakhotiya, Vaibhav Kumar, Ashwin Pathak, Raj Shah


Abstract
Large Language Models have been successful in a wide variety of Natural Language Processing tasks by capturing the compositionality of the text representations. In spite of their great success, these vector representations fail to capture meaning of idiomatic multi-word expressions (MWEs). In this paper, we focus on the detection of idiomatic expressions by using binary classification. We use a dataset consisting of the literal and idiomatic usage of MWEs in English and Portuguese. Thereafter, we perform the classification in two different settings: zero shot and one shot, to determine if a given sentence contains an idiom or not. N shot classification for this task is defined by N number of common idioms between the training and testing sets. In this paper, we train multiple Large Language Models in both the settings and achieve an F1 score (macro) of 0.73 for the zero shot setting and an F1 score (macro) of 0.85 for the one shot setting. An implementation of our work can be found at https://github.com/ashwinpathak20/Idiomaticity_Detection_Using_Few_Shot_Learning.
Anthology ID:
2022.semeval-1.19
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
165–168
Language:
URL:
https://aclanthology.org/2022.semeval-1.19
DOI:
10.18653/v1/2022.semeval-1.19
Bibkey:
Cite (ACL):
Yash Jakhotiya, Vaibhav Kumar, Ashwin Pathak, and Raj Shah. 2022. JARVix at SemEval-2022 Task 2: It Takes One to Know One? Idiomaticity Detection using Zero and One-Shot Learning. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 165–168, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
JARVix at SemEval-2022 Task 2: It Takes One to Know One? Idiomaticity Detection using Zero and One-Shot Learning (Jakhotiya et al., SemEval 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.semeval-1.19.pdf
Video:
 https://aclanthology.org/2022.semeval-1.19.mp4
Code
 ashwinpathak20/idiomaticity_detection_using_few_shot_learning