Subword tokenization determines a language model’s symbol inventory and sequence length, yet standard BPE selects merges by frequency alone. We test whether association-reweighted merges help on BabyLM Strict-Small. Holding the merge budget and architecture fixed and approximately matching word exposure, we train 97M-parameter causal decoders with three seeds using standard BPE, the S-BPE merge score with fixed-budget stopping, and exponential positive-PMI reweighting (PMI-BPE). Neither alternative reliably improves held-out bits-per-byte (BPB) or BLiMP. PMI-BPE’s mean BPB is slightly higher than standard BPE’s, but no pairwise difference is statistically reliable at three seeds. Both alternatives alter the merge tables and measurably change compression, ruling out a trivial no-intervention explanation. Yet intrinsic association rankings depend on scoring measure and aggregation. Type-uniform rankings reverse between post-merge PMI and thresholded held-out PPMI, while application-weighted PPMI is nearly identical across tokenizers. These results do not rule out association-aware tokenization; they cover one merge budget, one operating point per score, and two evaluation axes. Within that scope, the tested scores do not reliably help, and intrinsic merge association does not consistently align with the BPB and BLiMP point-estimate orderings.

Hidden-state probes have been used to predict factual errors in language-model outputs. We study whether the same representations also distinguish four types of prompt-model relationship: known, unknown, ambiguous, and misleading (built on a false premise). On a controlled 200-prompt dataset constructed to balance six surface features, all six pairwise type contrasts are linearly recoverable from hidden states at the position where generation begins, at AUROC ≥ 0.986 on three open-weight instruction-tuned models spanning two families and a fourfold parameter range. The probe exceeds surface-feature, TF-IDF, and frozen sentence-embedding baselines on every pair. Under group-aware evaluation on paraphrased prompts with group-level shuffled controls, probe AUROC remains at least 0.949 on every pair while lexical baselines degrade. These findings frame hallucination as one downstream consequence of a four-type representational structure that also captures ambiguity and false premises.