BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction

Xiao Zhou, Yilun Zhao, Owen Jiang, Tiansheng Hu, Cai Xu, Manasi Patwardhan, Arman Cohan


Abstract
Scientific weak signals are early, low-visibility research directions that later become central to mature scientific topics, yet existing resources such as trend tracking, citation forecasting, and foresight reports rarely provide validated reference sets that link concrete early precursors to later paradigms. We introduce BackTrend, a retrospective benchmark in which, given a mature target topic and a temporal evidence constraint, systems must recover two types of precursors: problem-space signals, underrecognized research problems, and solution-space signals, emerging methods for known problems. BackTrend contains 25 mature target topics in artificial intelligence and machine learning and 66 human-validated weak signals, reconstructed from large-scale literature by grounding each candidate in its 2019–2024 publication-frequency trajectory. We evaluate frontier LLMs, RAG systems, and agentic research systems via set-level and signal-level semantic matching. Current systems generate plausible but misaligned precursors, with topic drift, granularity mismatch, lexical near-miss, and coverage failure; no system exceeds single-digit LLM-judged F1, and additional retrieval or web search does not consistently help.
Anthology ID:
2026.findings-emnlp.622
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
13065–13094
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.622/
DOI:
Bibkey:
Cite (ACL):
Xiao Zhou, Yilun Zhao, Owen Jiang, Tiansheng Hu, Cai Xu, Manasi Patwardhan, and Arman Cohan. 2026. BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 13065–13094, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction (Zhou et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.622.pdf