2026

Static benchmarks often conflate memorization with reasoning, failing to capture the dynamic nature of world knowledge. We present LiveSearchBench, a scalable pipeline that automatically constructs snapshot-grounded benchmarks from knowledge graph differentials, in which every answer is unique and SPARQL-verifiable against a target snapshot. Unlike prior template-based approaches, our method leverages topology-aware synthesis to generate multi-constraint compositional questions with unique answers guaranteed via SPARQL validation. Evaluating 1,000 questions across three difficulty levels on both open-weight and frontier models, we expose a pronounced Recency Gap on facts that post-date a model’s training cutoff, particularly for compositional queries; retrieval dependence is thus an empirical property we measure, not a formal guarantee we claim. Through Oracle and Wikidata-as-corpus analyses, we disentangle dual failure modes: retrieval coverage limitations for single-hop questions and reasoning failures for compositional queries even with ground-truth evidence. LiveSearchBench provides both a rigorous construction methodology and a continuously updatable testbed for evaluating evidence integration under evolving knowledge.

2025

Multi-agent systems have emerged as a promising approach for enhancing the reasoning capabilities of large language models in complex problem-solving. However, current MAS frameworks are limited by poor flexibility and scalability, with underdeveloped optimization strategies. To address these challenges, we propose ReSo, which integrates task graph generation with a reward-driven two-stage agent selection process. The core of ReSo is the proposed Collaborative Reward Model, which can provide fine-grained reward signals for MAS cooperation for optimization. We also introduce an automated data synthesis framework for generating MAS benchmarks, without human annotations. Experimentally, ReSo matches or outperforms existing methods. ReSo achieves 33.7% and 32.3% accuracy on Math-MAS and SciBench-MAS SciBench, while other methods completely fail. The code and data are available at Reso.