Authorship attribution is the process of identifying the writer of a disputed text by analyzing distinctive stylistic patterns. By comparing the disputed text against a baseline of known authorship, the task amounts to assessing how strongly the two texts are likely to share an author. However, topical confounding makes the task deceptively easy: two authors covering the same topic may look more alike than one author covering two topics. Scholarly prose offers a natural solution. Academic writers produce multiple papers on related but distinct topics while maintaining consistent stylistic habits. Thus, we introduce HALvest-Contrastive, an English benchmark that draws same-author passages from distinct papers within a disciplinary field to reduce topical overlap, while using different-author passages from the same field to discourage naive keyword matching. The benchmark’s design removes topical shortcuts: a strong lexical baseline fails under these constraints. As a by-product of our benchmark construction, we release the 17B-token multilingual corpus used to produce it. We then revisit how to score authorship: while standard systems compress each document into a single vector, we retain the sequence of token-level representations produced by a language model and compare them with late interaction. We also study patch-level late interaction, which groups neighboring tokens into patches before interaction. Matching at the token level increases Recall@20 by a factor of three to five. The optimal interaction granularity, however, depends on sequence length, with no single granularity performing best across all settings.

Authorship attribution asks whether two texts were written by the same author. Models fine-tuned for this task with the same pretrained encoder, data, and loss can still differ four-fold in Recall@20 depending only on their scoring mechanism. We use mechanistic interpretability tools to explain this gap. Stylistic features such as word length, punctuation density, and function-word frequency are similarly available at every layer in every model we probe, including an off-the-shelf control encoder, suggesting that differences in feature availability do not explain the gap. Instead, causal intervention shows that the scoring mechanism determines where the encoder consolidates authorship signal. Mean pooling forces consolidation in the middle third of the network, while late interaction defers it to the upper third. We further derive this difference from the gradient structure of each scorer, and training dynamics reveal distinct learning trajectories that follow from that difference.