Socrates Osorio

Author directory

2026

As post-training data can induce many unexpected behaviors in large language models (LLMs), scalable data attribution is important for efficiently tracing these behaviors back to the data that induced them. Most existing methods are expensive because they require running at minimum forward and backward passes (and sometimes Hessians) across the entire training corpus for each eval behavior. This makes it difficult to adaptively study new behaviors because each one requires training-sized compute. We propose a general framework for data attribution that generalizes recent approaches, which we observe to share a two-stage structure: training examples and behaviors are mapped independently to vectors, and attribution is a similarity between them. We use this to propose several novel activation-based or gradient-based methods. Evaluating their performance on Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, we find no single method performs best across behaviors. On free-generation tasks, two methods in our framework often outperform TRAK and persona-vector baselines, while a steering-guided mixture of the two almost always does. On reading comprehension hallucination, one gradient based method substantially outperforms the baselines, while the remaining methods and baselines perform similarly to random selection.