Magy Seif El-Nasr
Author directory2026
Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies
Reza Habibi | Darian Lee | Magy Seif El-Nasr
Findings of the Association for Computational Linguistics: EMNLP 2026
Reza Habibi | Darian Lee | Magy Seif El-Nasr
Findings of the Association for Computational Linguistics: EMNLP 2026
While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the behavioral decisions a model learns. We introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes training pairs to the interpretable symbolic policies governing model behavior. SMDA fits a closed-form Ridge regression over sparse autoencoder (SAE) features to model a behavior, then decomposes how each supervised fine-tuning (SFT) example shifts that policy through feature-activation (ΔX) and output-probability (ΔY) pathways. We distill a symbolic policy for refusal in Llama-3.2-3B-Instruct and analyze 200 SFT pairs. Our analysis reveals (1) the symbolic policy’s coefficients expose systematic gaps in the base model’s safety behavior, such as religious stereotyping; (2) per-feature ΔX/ΔY decomposition mechanistically explains why harmful and harmless pairs exert qualitatively different influences on certain features; and (3) individual pairs routinely exhibit cross-feature interference, allowing SMDA to identify pairs whose dominant effect falls on unintended features. Retraining without such pairs improved accuracy by up to 4.5 percentage points over full fine-tuning. These results show that combining mechanistic interpretability with data attribution yields a diagnostic tool both more fine-grained than black-box influence functions and more scalable than manual circuit analysis.