SQL code completion boosts developer productivity by predicting missing snippets from editing SQL contexts. However, existing solutions yield low acceptance since they cannot abstain when no suggestion is needed and lack SQL-specific training. To address both limitations, we introduce PACE-SQL, a preference-aligned framework that improves SQL completion accuracy and user preference alignment. We build two datasets with completion and abstention samples: WeSQL-1.5M from real-world interaction logs to capture authentic user preferences, and OpenSQL-FIM from public SQL corpora. We adopt continual pretraining and supervised fine-tuning to improve completion quality, and design a reward function in the reinforcement learning to enhance alignment with user preferences. Experiments show PACE-SQL improves Exact Match and Edit Similarity scores by 9.67% and 7.52% over the best baseline on WeSQL-1.5M and raises online user acceptance rate from 5% to 29%.

The surge in Large Language Models (LLMs) has revolutionized natural language processing, but fine-tuning them for specific tasks often encounters challenges in balancing performance and preserving general instruction-following abilities. In this paper, we posit that the distribution gap between task datasets and the LLMs serves as the primary underlying cause. To address the problem, we introduce Self-Distillation Fine-Tuning (SDFT), a novel approach that bridges the distribution gap by guiding fine-tuning with a distilled dataset generated by the model itself to match its original distribution. Experimental results on the Llama-2-chat model across various benchmarks demonstrate that SDFT effectively mitigates catastrophic forgetting while achieving comparable or superior performance on downstream tasks compared to the vanilla fine-tuning. Moreover, SDFT demonstrates the potential to maintain the helpfulness and safety alignment of LLMs. Our code is available at https://github.com/sail-sg/sdft.