Yonghong Yu
2025
PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing
Yiwen Duan
|
Yonghong Yu
|
Xiaoming Zhao
|
Yichang Wu
|
Wenbo Liu
Proceedings of the 31st International Conference on Computational Linguistics: Industry Track
Code Large Language Models (Code LLMs), such as Code llama and DeepSeek-Coder, have demonstrated exceptional performance in the code generation tasks. However, most existing models focus on the abilities of generating correct code, but often struggle with bug repair. We introduce a suit of methods to enhance LLM’s SQL bug-fixing abilities. The methods are mainly consisted of two parts: A Progressive Dataset Construction (PDC) from scratch and Dynamic Mask Supervised Fine-tuning (DM-SFT). PDC proposes two data expansion methods from the perspectives of breadth first and depth first respectively. DM-SFT introduces an efficient bug-fixing supervised learning approach, which effectively reduce the total training steps and mitigate the “disorientation” in SQL code bug-fixing training. In our evaluation, the code LLM models trained with two methods have exceeds all current best performing model which size is much larger.
2019
The LAIX Systems in the BEA-2019 GEC Shared Task
Ruobing Li
|
Chuan Wang
|
Yefei Zha
|
Yonghong Yu
|
Shiman Guo
|
Qiang Wang
|
Yang Liu
|
Hui Lin
Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications
In this paper, we describe two systems we developed for the three tracks we have participated in the BEA-2019 GEC Shared Task. We investigate competitive classification models with bi-directional recurrent neural networks (Bi-RNN) and neural machine translation (NMT) models. For different tracks, we use ensemble systems to selectively combine the NMT models, the classification models, and some rules, and demonstrate that an ensemble solution can effectively improve GEC performance over single systems. Our GEC systems ranked the first in the Unrestricted Track, and the third in both the Restricted Track and the Low Resource Track.