Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing

Hyeonchu Park, Gahye Jeong, Bugeun Kim


Abstract
AI text detectors are increasingly employed in academic settings, but it remains unclear whether their outputs reflect AI authorship itself or broader linguistic features associated with polished academic English. Previous studies have reported high false-positive rates (FPRs) for non-native English writing, but population-level comparisons confound authorship with differences in topic, domain, and writing style. Professional editing provides a useful setting for examining this issue because it changes the linguistic form of manuscripts while preserving authorship and content. We examined 135,389 document pairs from a professional English editing service (2018–2025), comprising non-native manuscripts and their native-edited versions, to assess how editing affects detector responses controlling for content and authorship. For the 13 AI text detectors, FPRs for human-written texts varied widely, from 0.0% to 100.0%. Responses varied across detectors: the same edits increased AI scores in some detectors but decreased them in others. Notably, score changes correlated with the extent of editing. The findings identify professional editing style as a key confounding variable in AI detector outputs, rather than establishing a full separation of text origin from linguistic style, raising concerns about fairness and reliability in academic settings.
Anthology ID:
2026.emnlp-industry.109
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1872–1887
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.109/
DOI:
Bibkey:
Cite (ACL):
Hyeonchu Park, Gahye Jeong, and Bugeun Kim. 2026. Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1872–1887, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing (Park et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.109.pdf