Large Language Models (LLMs) are transforming chemical research by enabling specialized reasoning models and tool-augmented agents. Yet existing benchmarks typically evaluate reasoning or tool invocation in isolation, overlooking the dynamic interaction between internal deliberation and external execution. We present ChemEval2, a multimodal benchmark for tool-augmented chemical reasoning, containing 2,410 samples spanning fundamental principles, spectroscopic analysis, and complex reaction understanding. Built from professional textbooks and experimental data through automated generation and rigorous human annotation, ChemEval2 introduces a tool-augmented Chain-of-Thought evaluation framework that embeds tool-use checkpoints at stages where unaided language reasoning becomes unreliable. This design evaluates whether models can identify information gaps, invoke appropriate tools, and integrate tool outputs into reasoning trajectories. Across diverse text and multimodal models, we observe a process–outcome inconsistency: models can satisfy intermediate reasoning steps or execute useful tool calls yet still fail at final synthesis. These results provide evidence that coordinating computational tool use with downstream chemical reasoning remains a substantial challenge. We open-source ChemEval2 and an MCP-based suite of specialized chemical tools for reproducible.

Recent advancements in large language models (LLMs) have revolutionized natural language processing through their remarkable capabilities in understanding and executing diverse tasks. While supervised fine-tuning, particularly in Retrieval-Augmented Generation (RAG) scenarios, effectively enhances task-specific performance, it often leads to catastrophic forgetting, where models lose their previously acquired knowledge and general capabilities. Existing solutions either require access to general instruction data or face limitations in preserving the model’s original distribution. To overcome these limitations, we propose SelfAug, a self-distribution alignment method that aligns input sequence logits to preserve the model’s semantic distribution, thereby mitigating catastrophic forgetting and improving downstream performance. Extensive experiments demonstrate that SelfAug achieves a superior balance between downstream learning and general capability retention. Our comprehensive empirical analysis reveals a direct correlation between distribution shifts and the severity of catastrophic forgetting in RAG scenarios, highlighting how the absence of RAG capabilities in general instruction tuning leads to significant distribution shifts during fine-tuning. Our findings not only advance the understanding of catastrophic forgetting in RAG contexts but also provide a practical solution applicable across diverse fine-tuning scenarios.