Machine-generated text is becoming a routine part of the web. Most detectors, however, are still evaluated on curated datasets rather than real web content. We introduce PAWN++, a compact supervised detector that learns which likelihood signals from two frozen language models are most informative. On MAGE, our method reaches 98.36 AUROC in distribution and 98.42 under a joint domain and generator shift. It also transfers well to external benchmarks without target-specific training.

To test whether this robustness extends to the web, we construct WATTLE, a benchmark of 41,300 texts in which each machine output is tied to its original human web page. The resulting data capture realistic transformations of web content as well as variation across languages and time. We use WATTLE to compare our method with seven other detectors. Performance deteriorates sharply on realistic rewrites: our method reaches 70.1 AUROC overall and 58.1 on text produced by a frontier model. Likelihood-based detectors fall below chance, and no evaluated system identifies more than 12% of frontier rewrites at a 1% false-positive rate. These results expose a substantial gap between standard evaluation scores and reliable detection on the web.

Safety benchmarks for large language models (LLMs) commonly use multiple-choice question (MCQ) format. However, prior studies documented that MCQ format reduces refusal rates, which may significantly affect the safety behavior of the model. We provide a first mechanistic explanation of this effect on the Weapons of Mass Destruction Proxy (WMDP) dataset across two models – Llama-3.1-8B and Qwen3-8B. First, we use the tuned lens technique to show that when processing MCQ the model commits to an answer-letter token very quickly, which is in contrast to the gradual refusal build-up in open-ended formats. Additionally, we observe that activations on the MCQ format lie roughly 3× further from the refusal direction, compared with their open-ended counterparts. Moreover, ablating the attention heads that carry the MCQ format answer increases the refusal, which is a direct evidence that the MCQ answer-selection competes with refusal. These findings indicate that MCQ safety scores primarily measure knowledge capabilities and do not assess the safety-mechanisms.