Yutao Fan
Author directory2026
An Automated Pipeline for Provably Retrieval-Dependent Benchmark Construction over Dynamic Knowledge
Heng Zhou | Ao Yu | Yuchen Fan | Li Kang | Jianing Shi | Yongting Zhang | Yutao Fan | Tiancheng He | Yibing Lin | Hejia Geng | Yuhao Wu | Xiufeng Song | Zhemeng Zhang | Songtao Huang | Yiran Qin | Wenlong Zhang | Lei Bai | Zhenfei Yin
Findings of the Association for Computational Linguistics: EMNLP 2026
Heng Zhou | Ao Yu | Yuchen Fan | Li Kang | Jianing Shi | Yongting Zhang | Yutao Fan | Tiancheng He | Yibing Lin | Hejia Geng | Yuhao Wu | Xiufeng Song | Zhemeng Zhang | Songtao Huang | Yiran Qin | Wenlong Zhang | Lei Bai | Zhenfei Yin
Findings of the Association for Computational Linguistics: EMNLP 2026
Static benchmarks often conflate memorization with reasoning, failing to capture the dynamic nature of world knowledge. We present LiveSearchBench, a scalable pipeline that automatically constructs snapshot-grounded benchmarks from knowledge graph differentials, in which every answer is unique and SPARQL-verifiable against a target snapshot. Unlike prior template-based approaches, our method leverages topology-aware synthesis to generate multi-constraint compositional questions with unique answers guaranteed via SPARQL validation. Evaluating 1,000 questions across three difficulty levels on both open-weight and frontier models, we expose a pronounced Recency Gap on facts that post-date a model’s training cutoff, particularly for compositional queries; retrieval dependence is thus an empirical property we measure, not a formal guarantee we claim. Through Oracle and Wikidata-as-corpus analyses, we disentangle dual failure modes: retrieval coverage limitations for single-hop questions and reasoning failures for compositional queries even with ground-truth evidence. LiveSearchBench provides both a rigorous construction methodology and a continuously updatable testbed for evaluating evidence integration under evolving knowledge.
LatticeMind: A Conflict-Aware Memory Primitive for Multi-Agent Systems
Heng Zhou | Lian Zhang | Yutao Fan | Tiancheng He | Siki Chen | Hejia Geng | Philip Torr | Zhenfei Yin
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Heng Zhou | Lian Zhang | Yutao Fan | Tiancheng He | Siki Chen | Hejia Geng | Philip Torr | Zhenfei Yin
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multi-agent LLM systems often fail not for lack of candidate answers, but because they have no persistent mechanism for deciding which incompatible claim should currently be trusted. Majority vote, debate, and judge-based selection choose an output without recording which claim wins, which is contested, or why a later update supersedes it. We present LatticeMind, a conflict-aware structured memory that handles contradiction at write time. It maintains explicit item status, applies cheap symbolic conflict checks, and routes the flagged cases to an LLM reconciler. On a label-blind ConflictBank evaluation that removes source-name hints, over a uniform draw of 200 examples from the full benchmark pool, LatticeMind reaches 0.965 accuracy versus 0.525 for the strongest baseline, with every gap significant at p<10-24 by paired McNemar test. Ablations show that removing the checker or the reconciler costs 12 to 14 points. Our claims are scoped to credibility conflicts, where one claim should defeat another: the coordination branch of the design is a safety override that we do not validate empirically, and all prompt-based comparisons are conditioned on a single backbone. On three secondary planning benchmarks and one synthetic scheduling diagnostic the picture is mixed: LatticeMind beats naive merge on three of four, but does not replace deliberation methods on tasks rewarding iterative search.
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
Zelin Tan | Hejia Geng | Xiaohang Yu | Mulei Zhang | Guancheng Wan | Yifan Zhou | Qiang He | Xiangyuan Xue | Heng Zhou | Yutao Fan | Zhong-Zhi Li | Zaibin Zhang | Guibin Zhang | Chen Zhang | Zhenfei Yin | Philip Torr | Lei Bai
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zelin Tan | Hejia Geng | Xiaohang Yu | Mulei Zhang | Guancheng Wan | Yifan Zhou | Qiang He | Xiangyuan Xue | Heng Zhou | Yutao Fan | Zhong-Zhi Li | Zaibin Zhang | Guibin Zhang | Chen Zhang | Zhenfei Yin | Philip Torr | Lei Bai
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
While scaling laws for large language models (LLMs) during pre-training have been extensively studied, their behavior under reinforcement learning (RL) post-training remains largely unexplored. This paper investigates the scaling behavior of Large Language Model (LLM) reinforcement learning post-training, focusing on mathematical reasoning. Through experiments across the Qwen2.5 series (0.5B to 72B), we characterize how model scale, data, and compute interact. Our analysis yields four key findings: 1. Larger models consistently demonstrate superior compute and data efficiency. 2. The relationship between model performance and training resources follows a predictive power-law across both base and instruction-tuned models. 3. RL learning efficiency exhibits a latent saturation trend with increasing model scale. 4. In data-constrained regimes, performance is primarily driven by the total volume of training data rather than sample uniqueness. These results offer practical guidelines for scaling reasoning capabilities through reinforcement learning post-training.
Search
Fix author
Co-authors
- Hejia Geng 3
- Zhenfei Yin 3
- Heng Zhou 3
- Lei Bai 2
- Tiancheng He 2
- Philip Torr 2
- Siki Chen 1
- Yuchen Fan 1
- Qiang He 1
- Songtao Huang 1
- Li Kang 1
- Zhong-Zhi Li 1
- Yibing Lin 1
- Yiran Qin 1
- Jianing Shi 1
- Xiufeng Song 1
- Zelin Tan 1
- Guancheng Wan 1
- Yuhao Wu 1
- Xiangyuan Xue 1
- Ao Yu 1
- Xiaohang Yu 1
- Chen Zhang 1
- Guibin Zhang 1
- Lian Zhang 1
- Mulei Zhang 1
- Wenlong Zhang 1
- Yongting Zhang 1
- Zaibin Zhang 1
- Zhemeng Zhang 1
- Yifan Zhou 1