Data attribution traces model behavior back to individual training examples, with applications such as identifying training data that induces toxic responses or tracing a model’s factual claims back to the examples that support them. Gradient-based influence methods provide reliable attribution performance, yet are impractical at the scale of LLM training datasets due to their high computational costs. We introduce Influcoder, which distills the LoRA-gradient cosine influence rankings of a target decoder into a compact encoder, reducing attribution at inference time to an embedding cosine similarity. It scores training examples 35x faster than gradient-based influence computed on the target model and 18x faster than influence computed on the smallest comparable proxy model. Our method remains competitive with gradient-based methods even when considering the setup cost, as evaluated on two downstream tasks of toxicity filtering and factual attribution.