Out-of-distribution (OOD) generalisation is essential for authorship attribution (AA), especially when involving new authors, yet this aspect remains insufficiently supported by existing AA datasets. In this paper, we introduce GHOSTWRITEBENCH, a dataset for LLM AA task, designed to test generalisation across multiple OOD dimensions, including unseen domain and authors. GHOSTWRITEBENCH comprises long-form texts (50K+ words per book) generated by ten frontier LLMs. We find that existing methods degrade substantially in unseen author scenarios — dropping by as much as 70% even under high-resource settings. We propose TRACE, a novel lightweight and interpretable fingerprinting method that captures token-level transition patterns (i.e., word rank and entropy) using a lightweight language model. Experiments show robustness of TRACE in OOD settings and establish it as a strong baseline on GHOSTWRITEBENCH.

With the growing integration of AI in daily life, ensuring the robustness of systems to inference-time attacks is crucial. Among the approaches for certifying robustness to such adversarial examples, randomized smoothing has emerged as highly promising due to its nature as a wrapper around arbitrary black-box models. Previous work on randomized smoothing in natural language processing has primarily focused on specific subsets of edit distance operations, such as synonym substitution or word insertion, without exploring the certification of all edit operations. In this paper, we adapt Randomized Deletion (Huang et al., 2023) and propose, CERTified Edit Distance defense (CERT-ED) for natural language classification. Through comprehensive experiments, we demonstrate that CERT-ED outperforms the existing Hamming distance method RanMASK (Zeng et al., 2023) in 4 out of 5 datasets in terms of both accuracy and the cardinality of the certificate. By covering various threat models, including 5 direct and 5 transfer attacks, our method improves empirical robustness in 38 out of 50 settings.