Daniel Little
Author directory2026
Belief or Circuitry? Causal Evidence for In-Context Graph Learning
Katharine Kowalyshyn | Timothy Duggan | Daniel Little | Michael C Hughes
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Katharine Kowalyshyn | Timothy Duggan | Daniel Little | Michael C Hughes
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
How do LLMs learn in-context? Is it by pattern-matching recent tokens, or by inferring latent structure? We probe this question using a toy graph random-walk across two competing graph structures. We present behavioral, representational, and causal evidence that neither account alone is sufficient. First, a complexity-weighted sigmoid model captures topology-dependent evidence accumulation, and PCA shows that both graph topologies are visually recoverable at intermediate mixture ratios. Second, residual-stream activation patching and graph-difference steering causally intervene on this graph-family signal: late-layer patching almost fully transfers the clean graph preference, while linear steering moves predictions in the intended direction and fails under norm-matched and label-shuffled controls. An explicit transition-cache baseline reproduces the fitted asymmetry and held-out-edge transfer, however, so these results do not by themselves distinguish global inference from local copying. Taken together, our findings are consistent with a dual-mechanism account in which structure inference and induction circuits operate in parallel, but do not uniquely identify it.