How much of a language model’s apparent knowledge reflects general linguistic competence, and how much reflects memorisation of facts seen during pretraining? Disentangling the two is normally confounded by data contamination, because modern corpora overlap with the benchmarks used to evaluate them. We exploit a natural experiment that removes exact benchmark exposure by construction. TALKIE-1930 is a 13B-parameter transformer trained exclusively on approximately 260 billion tokens of English published before 1931; TALKIE-WEB is an architecture-, tokeniser-, and compute-matched twin trained on an equivalent volume of FineWeb. Because every modern NLP benchmark postdates 1930, any post-1930 capability TALKIE-1930 exhibits cannot arise from benchmark contamination. We study this model pair along three axes. First, syntactic competence transfers across eras with only a modest gap (BLiMP: 76.9% for TALKIE-1930 vs. 82.0% for TALKIE-WEB), and an OCR-noise ablation shows that this gap is not an artefact of character-level corruption in the historical corpus. Second, in-context learning is largely preserved by temporal isolation: TALKIE-1930 matches TALKIE-WEB on SST-2 (91.2% vs. 90.9% at k=32), exceeds it on MNLI (47.3% vs. 37.8%), and learns post-2006 genres it cannot have seen (tweet sentiment and emotion) with learning-rate slopes statistically indistinguishable from TALKIE-WEB’s. Third, linear probes recover the temporal provenance of a factual statement from hidden states at 98.2% accuracy (TALKIE-1930) and 99.5% (TALKIE-WEB); a three-way probe on TALKIE-1930 that adds a suspected-leaked class reaches 96.0% accuracy (chance approximately 33%), well above lexical and permutation baselines, though this figure is carried mainly by the temporal contrast and the leaked class is only partially recovered. Finally, we document a dissociation between representation and behaviour: although TALKIE-1930’s hidden states linearly encode temporal-boundary information, its greedy decodings do not, confabulating post-1930 facts with fluent, confident anachronisms such as dating the iPhone to 1876. We frame these findings as a controlled case study of a rare temporally isolated model pair, providing evidence that exact benchmark exposure is not necessary for in-context learning on modern tasks, and that base language models contain linearly decodable representations of the temporal provenance and, in part, the epistemic status of their knowledge that need not surface in their outputs.

Multimodal models that interpret screen recordings are usually fed raw sampled video frames,, a faithful but costly perception layer: each frame is a large image input, and dense coverage multiplies token usage. We ask whether a cheap, text-first index (on-screen OCR text plus per-state change descriptions, built once by a small local pipeline) is a better-value perception layer for the same model. On GUI-World, a released benchmark of GUI recordings with human-annotated multiple-choice questions and an objective exact-match metric, we compare the two in a controlled setting (same model, same questions), and replicate across two flagship models. On Claude Opus 4.8, answering from the index reaches 0.850 accuracy vs.0.902 for an 8-frame baseline (n=286) at 44.6% lower cost, and a lossless content compression cuts cost to 75.6% below frames with no measurable accuracy loss. On GPT-5.5 the trade-off is even stronger: the compressed index trails frames by only about 3% (0.874 vs.0.895) while using 82.8% fewer input tokens. The index is also stable across models, the compressed result differing by only about three points between them (0.843 on Opus 4.8, 0.874 on GPT-5.5). The honest summary is comparable accuracy at a fraction of the cost.