Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models

Weixian Waylon Li, Mengyu Wang, Tiejun Ma


Abstract
Backtesting large language models (LLMs) on historical financial data is unreliable when their pre-training data include the evaluated events. An LLM trained in 2024 may already encode how stocks moved during 2018-2020. We name this failure parametric look-ahead bias and propose FinCAD, an inference-time adaptation of Context-Aware Decoding that attenuates contributions from memorised historical outcomes without retraining. FinCAD pairs an adversarial bias-discovery pipeline that learns a model-specific memory-activating prior prompt with an entity- and date-adaptive rule that scales the CAD strength using a per-(entity, date) confidence signal. Across five 7-14B LLMs and five mega-cap equities, the largest model-level mean in-sample return correction is -67.1%. For the three larger models, 2025 out-of-sample returns remain within 8K and mean Sharpe within ±0.10 of baseline; mean general-benchmark accuracy remains positive or within -1.7 points for four of five models. On an eleven-model leaderboard, FinCAD raises the subset-averaged in-sample/out-of-sample Spearman correlation from +0.779 to +0.846, yielding rankings that are more closely aligned with post-cutoff performance.
Anthology ID:
2026.emnlp-main.510
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
11694–11713
Language:
URL:
https://aclanthology.org/2026.emnlp-main.510/
DOI:
Bibkey:
Cite (ACL):
Weixian Waylon Li, Mengyu Wang, and Tiejun Ma. 2026. Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 11694–11713, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models (Li et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.510.pdf