Sunghun Kim

Author directory

2026

Large language models (LLMs) have made rapid progress in code generation, yet their ability to verify whether a program is correct has not necessarily improved at the same pace. We identify a generation-verification gap in code LLMs, where scaling substantially improves test-passing code generation, while execution-free correctness reasoning improves much more slowly. This gap challenges existing self-rewarding approaches, which often implicitly assume that a stronger generator also serves as a better judge. To bridge this gap, we propose CoVera, an execution-grounded co-evolution framework that jointly trains a code LLM to generate code and to verify code correctness. In generator mode, the model produces candidate programs and receives execution-based rewards from available tests. In verifier mode, the same model reads a problem-program pair, produces a reasoning trace, and predicts a correctness judgment, which is rewarded according to agreement with execution-derived labels. The calibrated verifier is then used as an auxiliary self-verification signal for generation, while the evolving generator continuously supplies increasingly challenging candidates for verifier training. This co-evolution enables an on-policy curriculum for correctness reasoning and grounds self-verification in execution feedback, without requiring exhaustive test suites or golden reference solutions. Extensive experiments across multiple benchmarks and model scales show that CoVera improves both code generation and intrinsic correctness verification, outperforming representative baselines from diverse code-learning paradigms.
While Large Language Models (LLMs) have revolutionized code generation, standard “System 1” approaches that generate solutions in a single forward pass often hit a performance ceiling on complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model’s weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-only training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B-14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, efficient reasoning and reflection patterns. The source code and data are available at https://github.com/juyongjiang/ReflexiCoder.
Large language models and coding agents can produce web applications that appear functional, yet surface plausibility is an unreliable proxy for behavioral conformance under interaction. We introduce ConformWeb, a contract-grounded protocol and benchmark that operationalizes public behavioral contracts to construct hidden interaction scenarios, evaluating whether agent-generated applications preserve strict conformance under dynamic execution. By decoupling hidden interaction sequences from the public behavioral scope, the protocol guarantees a fair held-out workflow evaluation without relying on undisclosed reference implementations. Evaluated across 24 validated instances scaling to 756 evaluation scenarios and 6,433 semantic steps, only 252 of the 2,160 total execution runs achieve full conformance, yielding a strict pass rate of 11.7%. While scenario-level conformance reaches 36.7% and step-level progress averages 49.4%, this quantitative gap demonstrates that generative systems frequently achieve substantial partial progress but fail to maintain the critical behavioral invariants required for end-to-end conformance. All code and data are released at https://github.com/deep-diver/ConformWeb.

2024

We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, demonstrating superior performance in various natural language processing (NLP) tasks. Inspired by recent efforts to efficiently up-scale LLMs, we present a method for scaling LLMs called depth up-scaling (DUS), which encompasses depthwise scaling and continued pretraining. In contrast to other LLM up-scaling methods that use mixture-of-experts, DUS does not require complex changes to train and inference efficiently. We show experimentally that DUS is simple yet effective in scaling up high-performance LLMs from small ones. Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, a variant fine-tuned for instruction-following capabilities, surpassing Mixtral-8x7B-Instruct. SOLAR 10.7B is publicly available under the Apache 2.0 license, promoting broad access and application in the LLM field.

2019

Generating SQL codes from natural language questions (NL2SQL) is an emerging research area. Existing studies have mainly focused on clear scenarios where specified information is fully given to generate a SQL query. However, in developer forums such as Stack Overflow, questions cover more diverse tasks including table manipulation or performance issues, where a table is not specified. The SQL query posted in Stack Overflow, Pseudo-SQL (pSQL), does not usually contain table schemas and is not necessarily executable, is sufficient to guide developers. Here we describe a new NL2pSQL task to generate pSQL codes from natural language questions on under-specified database issues, NL2pSQL. In addition, we define two new metrics suitable for the proposed NL2pSQL task, Canonical-BLEU and SQL-BLEU, instead of the conventional BLEU. With a baseline model using sequence-to-sequence architecture integrated by denoising autoencoder, we confirm the validity of our task. Experiments show that the proposed NL2pSQL approach yields well-formed queries (up to 43% more than a standard Seq2Seq model). Our code and datasets will be publicly released.