Bradly Stadie
Author directory2026
All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting
Zeyu Zhang | Bradly Stadie
Findings of the Association for Computational Linguistics: EMNLP 2026
Zeyu Zhang | Bradly Stadie
Findings of the Association for Computational Linguistics: EMNLP 2026
Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim’s decision impact, yielding Shapley-DCLR (Shapley-weighted Decision-Critical Leakage Rate)—an interpretable metric measuring what fraction of decision-driving reasoning is contaminated. We further propose TimeSPEC (Time-Supervised Prediction with Extracted Claims), an inference-time architecture that interleaves temporally-filtered retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm retrieval and supervision are jointly necessary; and a three-task probe further illustrates that the performance cost of temporal enforcement scales with each task’s reliance on post-cutoff information.