Zhilin Zhang

Author directory

2026

Text-to-SQL systems allow users to interact with databases using natural language, but most assume that each question must be answered by a single SQL query. In practical settings, some information needs are more reliably satisfied by generating multiple simpler SQL queries, which is not supported by existing benchmark or evaluation protocols. To address this gap, we introduce Spider𝕊, which, to the best of our knowledge, is the first benchmark for the Text-to-(Multi)-SQL task, supporting both single- and multi-query solutions within a unified evaluation framework. To enable this setting, we propose a flexible evaluation protocol that accepts any prediction covering the required information, regardless of query number or structure. Under our evaluation protocol, we evaluate several large language models on Spider𝕊 and find that allowing multi-query outputs improves execution accuracy. However, current models demonstrate fragility: even a minor prompt change from generating one SQL to generating one or more SQL can significantly degrade performance. To improve robustness, we introduce a lightweight router that decides whether to generate a single or multiple SQL for a given question, leading to substantial performance gains.