Clemens Vetter

Author directory

2026

Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attributes EM to persona features: latent directions representing misaligned personas, acquired during pre-training, are amplified during fine-tuning on misaligned data. We ask where these features come from, and whether naturally occurring human-written text suffices to induce EM. Using Sparse Autoencoder (SAE) based model diffing across four open-weight models, we find that misalignment fine-tuning amplifies features related to jailbreak personas, sarcasm, deception, and manipulation, while suppressing safety-relevant and assistant-identity features. Steering individual features controls EM in both directions: it induces misalignment rates of up to 62% in aligned models and re-aligns misaligned models to near-baseline rates. Attributing the causal features to one million web documents from an open pre-training corpus retrieves recurring narratives about villainous characters, domination, and harmful agency. Yet fine-tuning on these human-written documents does not reliably induce EM, whereas synthetic instruction-response pairs derived from the same content do, even across models. Semantic relevance alone is thus not sufficient: response structure or model-generated phrasing plays an important role in inducing EM.