CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions

Hanchong Zhang, Ruisheng Cao, Hongshen Xu, Lu Chen, Kai Yu


Abstract
Recently, Large Language Models (LLMs) have been demonstrated to possess impressive capabilities in a variety of domains and tasks. We investigate the issue of prompt design in the multi-turn text-to-SQL task and attempt to enhance the LLMs’ reasoning capacity when generating SQL queries. In the conversational context, the current SQL query can be modified from the preceding SQL query with only a few operations due to the context dependency. We introduce our method called CoE-SQL which can prompt LLMs to generate the SQL query based on the previously generated SQL query with an edition chain. We also conduct extensive ablation studies to determine the optimal configuration of our approach. Our approach outperforms different in-context learning baselines stably and achieves state-of-the-art performances on two benchmarks SParC and CoSQL using LLMs, which is also competitive to the SOTA fine-tuned models.
Anthology ID:
2024.naacl-long.361
Volume:
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6487–6508
Language:
URL:
https://aclanthology.org/2024.naacl-long.361
DOI:
10.18653/v1/2024.naacl-long.361
Bibkey:
Cite (ACL):
Hanchong Zhang, Ruisheng Cao, Hongshen Xu, Lu Chen, and Kai Yu. 2024. CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6487–6508, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions (Zhang et al., NAACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.naacl-long.361.pdf