Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards

Shihao Zhang, Xiaoman Wang, Yuan Liu, Yunshi Lan, Weining Qian


Abstract
Reinforcement learning has recently shown promise in improving large language models for Text-to-SQL generation, yet existing methods typically optimize one-shot rewards defined over a single SQL state. Such rewards provide limited guidance for iterative SQL correction and are insufficient to capture the improvement of multi-turn SQL refinement. In this paper, we propose Progress-SQL, a multi-turn reinforcement learning framework with progressive rewards for Text-to-SQL. Our approach introduces an Oracle-guided Diagnostic Tree (ODT), which abstracts SQL queries into clause-level structural profiles and produces diagnostic feedback for next-turn refinement. To provide dense and robust reward signals, we combine ODT-based structural alignment with lexical alignment and define a progressive reward that measures the improvement from the initial SQL to the final SQL. We further incorporate a progression latency reward that favors earlier correctness and an execution status reward that encourages recovery from the invalid SQL. Experiments on BIRD, Spider, and Spider robustness variants demonstrate that our method consistently improves Text-to-SQL performance across both primary and robustness evaluations.
Anthology ID:
2026.findings-emnlp.834
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
17675–17695
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.834/
DOI:
Bibkey:
Cite (ACL):
Shihao Zhang, Xiaoman Wang, Yuan Liu, Yunshi Lan, and Weining Qian. 2026. Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 17675–17695, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards (Zhang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.834.pdf