Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph

Usman Ahmed, Lubna Zafar, Faiza Qayyum, Muhammad Arshad Islam


Abstract
This paper describes the Irony detection system that participates in SemEval-2018 Task 3: Irony detection in English tweets. The system participated in the subtasks A and B. This paper discusses the results of our system in the development, evaluation and post evaluation. Each class in the dataset is represented as directed unweighted graphs. Then, the comparison is carried out with each class graph which results in a vector. This vector is used as features by machine learning algorithm. The model is evaluated on a hold on strategy. The organizers randomly split 80% (3,833 instances) training set (provided to the participant in training their system) and testing set 20%(958 instances). The test set is reserved to evaluate the performance of participants systems. During the evaluation, our system ranked 23 in the Coda Lab result of the subtask A (binary class problem). The binary class system achieves accuracy 0.6135, precision 0.5091, recall 0.7170 and F measure 0.5955. The subtask B (multi-class problem) system is ranked 22 in Coda Lab results. The multiclass model achieves the accuracy 0.4158, precision 0.4055, recall 0.3526 and f measure 0.3101.
Anthology ID:
S18-1095
Volume:
Proceedings of the 12th International Workshop on Semantic Evaluation
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marianna Apidianaki, Saif M. Mohammad, Jonathan May, Ekaterina Shutova, Steven Bethard, Marine Carpuat
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
581–586
Language:
URL:
https://aclanthology.org/S18-1095
DOI:
10.18653/v1/S18-1095
Bibkey:
Cite (ACL):
Usman Ahmed, Lubna Zafar, Faiza Qayyum, and Muhammad Arshad Islam. 2018. Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 581–586, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph (Ahmed et al., SemEval 2018)
Copy Citation:
PDF:
https://aclanthology.org/S18-1095.pdf