Evgenii Ptitsyn

Author directory

2026

Machine-generated text is becoming a routine part of the web. Most detectors, however, are still evaluated on curated datasets rather than real web content. We introduce PAWN++, a compact supervised detector that learns which likelihood signals from two frozen language models are most informative. On MAGE, our method reaches 98.36 AUROC in distribution and 98.42 under a joint domain and generator shift. It also transfers well to external benchmarks without target-specific training.

To test whether this robustness extends to the web, we construct WATTLE, a benchmark of 41,300 texts in which each machine output is tied to its original human web page. The resulting data capture realistic transformations of web content as well as variation across languages and time. We use WATTLE to compare our method with seven other detectors. Performance deteriorates sharply on realistic rewrites: our method reaches 70.1 AUROC overall and 58.1 on text produced by a frontier model. Likelihood-based detectors fall below chance, and no evaluated system identifies more than 12% of frontier rewrites at a 1% false-positive rate. These results expose a substantial gap between standard evaluation scores and reliable detection on the web.