LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXiv

Yanai Elazar, Maria Antoniak


Abstract
ArXiv recently prohibited the upload of unpublished review papers to its servers in the Computer Science domain, citing a high prevalence of LLM-generated content in these categories. However, this decision was not accompanied by quantitative evidence. In this work, we investigate this claim by measuring the proportion of LLM-generated content in review vs. non-review research papers in recent years. Using two high-quality detection methods, we find a substantial increase in LLM-generated content across both review and non-review papers, with a higher prevalence in review papers. However, when considering the number of LLM-generated papers published in each category, the estimates of non-review LLM-generated papers are almost six times higher. Furthermore, we find that this policy will affect papers in certain domains far more than others, with the CS subdiscipline Computers Society potentially facing cuts of 50%. Our analysis provides an evidence-based framework for evaluating such policy decisions.
Anthology ID:
2026.findings-emnlp.2365
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
50830–50848
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2365/
DOI:
10.18653/2026.findings-emnlp.2365
Bibkey:
Cite (ACL):
Yanai Elazar and Maria Antoniak. 2026. LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXiv. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 50830–50848, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXiv (Elazar & Antoniak, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2365.pdf