SQL code completion boosts developer productivity by predicting missing snippets from editing SQL contexts. However, existing solutions yield low acceptance since they cannot abstain when no suggestion is needed and lack SQL-specific training. To address both limitations, we introduce PACE-SQL, a preference-aligned framework that improves SQL completion accuracy and user preference alignment. We build two datasets with completion and abstention samples: WeSQL-1.5M from real-world interaction logs to capture authentic user preferences, and OpenSQL-FIM from public SQL corpora. We adopt continual pretraining and supervised fine-tuning to improve completion quality, and design a reward function in the reinforcement learning to enhance alignment with user preferences. Experiments show PACE-SQL improves Exact Match and Edit Similarity scores by 9.67% and 7.52% over the best baseline on WeSQL-1.5M and raises online user acceptance rate from 5% to 29%.

The unprecedented performance of LLMs requires comprehensive and accurate evaluation. We argue that for LLMs evaluation, benchmarks need to be comprehensive and systematic. To this end, we propose the Zhujiu benchmark, which has the following strengths: (1) Multi-dimensional ability coverage: We comprehensively evaluate LLMs across 7 ability dimensions covering 51 tasks. Especially, we also propose a new benchmark that focus on knowledge ability of LLMs. (2) Multi-faceted evaluation methods collaboration: We use 3 different yet complementary evaluation methods to comprehensively evaluate LLMs, which can ensure the authority and accuracy of the evaluation results. (3) Comprehensive Chinese benchmark: ZhuJiu is the pioneering benchmark that fully assesses LLMs in Chinese, while also providing equally robust evaluation abilities in English. (4) Avoiding potential data leakage: To avoid data leakage, we construct evaluation data specifically for 37 tasks. We evaluate 10 current mainstream LLMs, and conduct an in-depth discussion and analysis of their results. The ZhuJiu benchmark and open-participation leaderboard are publicly released at http://www.zhujiu-benchmark.com and we also provide a demo video at https://youtu.be/qypkJ89L1Ic.