UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation

Amirhossein Abaskohi, Arash Rasouli, Tanin Zeraati, Behnam Bahrak


Abstract
Sarcasm is a term that refers to the use of words to mock, irritate, or amuse someone. It is commonly used on social media. The metaphorical and creative nature of sarcasm presents a significant difficulty for sentiment analysis systems based on affective computing. The methodology and results of our team, UTNLP, in the SemEval-2022 shared task 6 on sarcasm detection are presented in this paper. We put different models, and data augmentation approaches to the test and report on which one works best. The tests begin with traditional machine learning models and progress to transformer-based and attention-based models. We employed data augmentation based on data mutation and data generation. Using RoBERTa and mutation-based data augmentation, our best approach achieved an F1-score of 0.38 in the competition’s evaluation phase. After the competition, we fixed our model’s flaws and achieved anF1-score of 0.414.
Anthology ID:
2022.semeval-1.135
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
962–969
Language:
URL:
https://aclanthology.org/2022.semeval-1.135
DOI:
10.18653/v1/2022.semeval-1.135
Bibkey:
Cite (ACL):
Amirhossein Abaskohi, Arash Rasouli, Tanin Zeraati, and Behnam Bahrak. 2022. UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 962–969, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
UTNLP at SemEval-2022 Task 6: A Comparative Analysis of Sarcasm Detection Using Generative-based and Mutation-based Data Augmentation (Abaskohi et al., SemEval 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.semeval-1.135.pdf
Video:
 https://aclanthology.org/2022.semeval-1.135.mp4
Code
 amirabaskohi/semeval2022-task6-sarcasm-detection +  additional community code
Data
Headlines datasetiSarcasm