Kai Jin
Author directory2026
From Atomic to Agentic: Towards Interpretable Evaluation of LLMs’ Agentic Mathematical Capabilities
Jiayi Kuang | Yinghui Li | Yunze Song | Keyu Chen | Zhifeng Shen | Yangning Li | Yidong Wang | di Yin | Ruizhi Qiao | Xing Sun | Kai Jin | Ying Shen | Liang Lin | Philip S. Yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiayi Kuang | Yinghui Li | Yunze Song | Keyu Chen | Zhifeng Shen | Yangning Li | Yidong Wang | di Yin | Ruizhi Qiao | Xing Sun | Kai Jin | Ying Shen | Liang Lin | Philip S. Yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large Language Models (LLMs) are evolving from performing end-to-end mathematical reasoning to integrating agentic intelligence. However, most existing math benchmarks evaluate only final answers. This outcome-oriented evaluation provides limited diagnostic value for identifying process-level failures or rigorous logic, failing to guide the transformation of LLMs into robust agents. To bridge this gap, we present a process-level benchmark designed to evaluate the inherent agentic mathematical reasoning abilities of LLMs. Our framework aligns problem-solving agentic behaviors with a structured taxonomy of reusable mathematical atomic capabilities. We design a comprehensive suite of planning, action, and feedback tasks across both textual and multimodal contexts, supported by an automated pipeline that synthesizes high-quality trajectories and produces fine-grained annotations via controlled LLM rewriting. Experiments reveal that models with stronger end-to-end accuracy can exhibit markedly different agentic capability profiles. This demonstrates that process-level evaluation is crucial for interpreting the true potential of LLMs and guiding the development of next-generation mathematical agents.
2024
Decremental Learning for Domain Adaptation in Neural Machine Translation
Kai Jin | Wuying Liu
Proceedings of the 36th Conference on Computational Linguistics and Speech Processing (ROCLING 2024)
Kai Jin | Wuying Liu
Proceedings of the 36th Conference on Computational Linguistics and Speech Processing (ROCLING 2024)
2023
Morphological and Semantic Evaluation of Ancient Chinese Machine Translation
Kai Jin | Dan Zhao | Wuying Liu
Proceedings of the Ancient Language Processing Workshop
Kai Jin | Dan Zhao | Wuying Liu
Proceedings of the Ancient Language Processing Workshop
Machine translation (MT) of ancient Chinese texts presents unique challenges due to the complex grammatical structures, cultural nuances, and polysemy of the language. This paper focuses on evaluating the translation quality of different platforms for ancient Chinese texts using The Analects as a case study. The evaluation is conducted using the BLEU, LMS, and ESS metrics, and the platforms compared include three machine translation platforms (Baidu Translate, Bing Microsoft Translator, and DeepL), and one language generation model ChatGPT that can engage in translation endeavors. Results show that Baidu performs the best, surpassing the other platforms in all three metrics, while ChatGPT ranks second and demonstrates unique advantages. The translations generated by ChatGPT are deemed highly valuable as references. The study contributes to understanding the challenges of MT for ancient Chinese texts and provides insights for users and researchers in this field. It also highlights the importance of considering specific domain requirements when evaluating MT systems.