Targeted Token-Level Unlearning for Repairing Text-to-SQL Models

Jinyu Zhang, Ruiheng Liu, Yu Zhang


Abstract
Text-to-SQL models may exhibit recurring error generation patterns that are reproducible under a fixed model and difficult to eliminate through continued training. Existing approaches either perform post-hoc correction at the output level or require access to the target domain’s training data for preference optimization, neither of which directly targets these reproducible local decision tendencies. To address these limitations, we propose Targeted Token-level Unlearning for SQL (TTU-SQL), a machine-unlearning-based framework that uses a probe dataset to detect and correct systematic errors without requiring access to the model’s original training data. The framework comprises three components: (1) AST-based token-level divergence detection that precisely localizes erroneous decision points; (2) a targeted error-type extraction procedure that selects three specific types of forgettable errors based on AST divergence and logit-level behavior; and (3) Part-NPO, which applies negative preference optimization only to localized divergence tokens to suppress erroneous continuations while preserving useful SQL generation knowledge. Across the Spider and BIRD datasets and all evaluated model backbones, TTU-SQL yields consistent execution-accuracy gains over equal-budget training-based repair baselines.
Anthology ID:
2026.emnlp-main.1345
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
30667–30687
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1345/
DOI:
Bibkey:
Cite (ACL):
Jinyu Zhang, Ruiheng Liu, and Yu Zhang. 2026. Targeted Token-Level Unlearning for Repairing Text-to-SQL Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 30667–30687, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Targeted Token-Level Unlearning for Repairing Text-to-SQL Models (Zhang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1345.pdf