Yunze Song
Author directory2026
From Atomic to Agentic: Towards Interpretable Evaluation of LLMs’ Agentic Mathematical Capabilities
Jiayi Kuang | Yinghui Li | Yunze Song | Keyu Chen | Zhifeng Shen | Yangning Li | Yidong Wang | di Yin | Ruizhi Qiao | Xing Sun | Kai Jin | Ying Shen | Liang Lin | Philip S. Yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiayi Kuang | Yinghui Li | Yunze Song | Keyu Chen | Zhifeng Shen | Yangning Li | Yidong Wang | di Yin | Ruizhi Qiao | Xing Sun | Kai Jin | Ying Shen | Liang Lin | Philip S. Yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large Language Models (LLMs) are evolving from performing end-to-end mathematical reasoning to integrating agentic intelligence. However, most existing math benchmarks evaluate only final answers. This outcome-oriented evaluation provides limited diagnostic value for identifying process-level failures or rigorous logic, failing to guide the transformation of LLMs into robust agents. To bridge this gap, we present a process-level benchmark designed to evaluate the inherent agentic mathematical reasoning abilities of LLMs. Our framework aligns problem-solving agentic behaviors with a structured taxonomy of reusable mathematical atomic capabilities. We design a comprehensive suite of planning, action, and feedback tasks across both textual and multimodal contexts, supported by an automated pipeline that synthesizes high-quality trajectories and produces fine-grained annotations via controlled LLM rewriting. Experiments reveal that models with stronger end-to-end accuracy can exhibit markedly different agentic capability profiles. This demonstrates that process-level evaluation is crucial for interpreting the true potential of LLMs and guiding the development of next-generation mathematical agents.