Fine-tuning Language Models for AI vs Human Generated Text detection

Sankalp Bahad, Yash Bhaskar, Parameswari Krishnamurthy


Abstract
In this paper, we introduce a machine-generated text detection system designed totackle the challenges posed by the prolifera-tion of large language models (LLMs). Withthe rise of LLMs such as ChatGPT and GPT-4,there is a growing concern regarding the po-tential misuse of machine-generated content,including misinformation dissemination. Oursystem addresses this issue by automating theidentification of machine-generated text acrossmultiple subtasks: binary human-written vs.machine-generated text classification, multi-way machine-generated text classification, andhuman-machine mixed text detection. We em-ploy the RoBERTa Base model and fine-tuneit on a diverse dataset encompassing variousdomains, languages, and sources. Throughrigorous evaluation, we demonstrate the effec-tiveness of our system in accurately detectingmachine-generated text, contributing to effortsaimed at mitigating its potential misuse.
Anthology ID:
2024.semeval-1.132
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
918–921
Language:
URL:
https://aclanthology.org/2024.semeval-1.132
DOI:
10.18653/v1/2024.semeval-1.132
Bibkey:
Cite (ACL):
Sankalp Bahad, Yash Bhaskar, and Parameswari Krishnamurthy. 2024. Fine-tuning Language Models for AI vs Human Generated Text detection. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 918–921, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Fine-tuning Language Models for AI vs Human Generated Text detection (Bahad et al., SemEval 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.semeval-1.132.pdf
Supplementary material:
 2024.semeval-1.132.SupplementaryMaterial.txt