Junying Wang

Author directory

2026

Tool-integrated reasoning has emerged as a promising paradigm for Text-to-SQL, framing SQL generation as sequential decision-making interleaved with tool execution. However, existing reinforcement learning approaches rely on coarse-grained outcome supervision, creating a credit assignment problem: models receive identical rewards regardless of whether intermediate steps are redundant or erroneous. We observe that tool-integrated reasoning offers a unique opportunity: each tool invocation produces structured, verifiable outputs that can serve as training-free step-level signals, eliminating the need for expensive reward model training. Building on this insight, we propose FineStep, a step-level credit assignment framework that leverages training-free process rewards derived from tool execution feedback. FineStep constructs a dual-channel value estimate for each step, combining backward-propagated outcome value with forward-smoothed process value, and synthesizes them via group-based normalization into a unified step-level advantage for policy optimization. Experiments on Text-to-SQL benchmarks show that FineStep achieves SOTA performance with a 3.25% EX gain over GRPO at the 4B scale, while results on AIME24 demonstrate transferability beyond SQL.