Sharif-MGTD at SemEval-2024 Task 8: A Transformer-Based Approach to Detect Machine Generated Text

Seyedeh Fatemeh Ebrahimi, Karim Akhavan Azari, Amirmasoud Iravani, Arian Qazvini, Pouya Sadeghi, Zeinab Taghavi, Hossein Sameti


Abstract
In this paper, we delve into the realm of detecting machine-generated text (MGT) within Natural Language Processing (NLP). Our approach involves fine-tuning a RoBERTa-base Transformer, a robust neural architecture, to tackle MGT detection as a binary classification task. Specifically focusing on Subtask A (Monolingual - English) within the SemEval-2024 competition framework, our system achieves a 78.9% accuracy on the test dataset, placing us 57th among participants. While our system demonstrates proficiency in identifying human-written texts, it faces challenges in accurately discerning MGTs.
Anthology ID:
2024.semeval-1.85
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
565–572
Language:
URL:
https://aclanthology.org/2024.semeval-1.85
DOI:
10.18653/v1/2024.semeval-1.85
Bibkey:
Cite (ACL):
Seyedeh Fatemeh Ebrahimi, Karim Akhavan Azari, Amirmasoud Iravani, Arian Qazvini, Pouya Sadeghi, Zeinab Taghavi, and Hossein Sameti. 2024. Sharif-MGTD at SemEval-2024 Task 8: A Transformer-Based Approach to Detect Machine Generated Text. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 565–572, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Sharif-MGTD at SemEval-2024 Task 8: A Transformer-Based Approach to Detect Machine Generated Text (Ebrahimi et al., SemEval 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.semeval-1.85.pdf
Supplementary material:
 2024.semeval-1.85.SupplementaryMaterial.txt
Supplementary material:
 2024.semeval-1.85.SupplementaryMaterial.zip