DAMAGE: Detecting Adversarially Modified AI Generated Text

Elyas Masrour, Bradley N. Emi, Max Spero


Abstract
AI humanizers are a new class of online software tools meant to paraphrase and rewrite AI-generated text in a way that allows them to evade AI detection software. We study 19 AI humanizer and paraphrasing tools and qualitatively assess their effects and faithfulness in preserving the meaning of the original text. We show that many existing AI detectors fail to detect humanized text. Finally, we demonstrate a robust model that can detect humanized AI text while maintaining a low false positive rate using a data-centric augmentation approach. We attack our own detector, training our own fine-tuned model optimized against our detector’s predictions, and show that our detector’s cross-humanizer generalization is sufficient to remain robust to this attack.
Anthology ID:
2025.genaidetect-1.9
Volume:
Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect)
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Firoj Alam, Preslav Nakov, Nizar Habash, Iryna Gurevych, Shammur Chowdhury, Artem Shelmanov, Yuxia Wang, Ekaterina Artemova, Mucahid Kutlu, George Mikros
Venues:
GenAIDetect | WS
SIG:
Publisher:
International Conference on Computational Linguistics
Note:
Pages:
120–133
Language:
URL:
https://aclanthology.org/2025.genaidetect-1.9/
DOI:
Bibkey:
Cite (ACL):
Elyas Masrour, Bradley N. Emi, and Max Spero. 2025. DAMAGE: Detecting Adversarially Modified AI Generated Text. In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 120–133, Abu Dhabi, UAE. International Conference on Computational Linguistics.
Cite (Informal):
DAMAGE: Detecting Adversarially Modified AI Generated Text (Masrour et al., GenAIDetect 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.genaidetect-1.9.pdf