Papilusion at DAGPap24: Paper or Illusion? Detecting AI-generated Scientific Papers

Nikita Andreev, Alexander Shirnin, Vladislav Mikhailov, Ekaterina Artemova


Abstract
This paper presents Papilusion, an AI-generated scientific text detector developed within the DAGPap24 shared task on detecting automatically generated scientific papers. We propose an ensemble-based approach and conduct ablation studies to analyze the effect of the detector configurations on the performance. Papilusion is ranked 6th on the leaderboard, and we improve our performance after the competition ended, achieving 99.46 (+9.63) of the F1-score on the official test set.
Anthology ID:
2024.sdp-1.20
Volume:
Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Tirthankar Ghosal, Amanpreet Singh, Anita Waard, Philipp Mayr, Aakanksha Naik, Orion Weller, Yoonjoo Lee, Shannon Shen, Yanxia Qin
Venues:
sdp | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
215–219
Language:
URL:
https://aclanthology.org/2024.sdp-1.20
DOI:
Bibkey:
Cite (ACL):
Nikita Andreev, Alexander Shirnin, Vladislav Mikhailov, and Ekaterina Artemova. 2024. Papilusion at DAGPap24: Paper or Illusion? Detecting AI-generated Scientific Papers. In Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), pages 215–219, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Papilusion at DAGPap24: Paper or Illusion? Detecting AI-generated Scientific Papers (Andreev et al., sdp-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.sdp-1.20.pdf