Jiasi Shen
Author directory2026
TeleGen: Improving LLM-Based Web Application Generation via Runtime Telemetry
Yujia Luo | Haonan Zhang | Jiasi Shen | Zishuo Ding | Weiyi Shang
Findings of the Association for Computational Linguistics: EMNLP 2026
Yujia Luo | Haonan Zhang | Jiasi Shen | Zishuo Ding | Weiyi Shang
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models can generate runnable web applications from natural-language requirements, but many generated applications still fail interactive tasks. Existing generate-execute-repair pipelines execute the generated application and use task outcomes or error messages to guide code revision. However, this feedback often misses the runtime behavior between a browser action and the final task outcome, making interaction-level failures difficult to diagnose. Therefore, we propose TeleGen, an observability-enhanced framework for LLM-based web application generation. TeleGen instruments generated applications, collects runtime telemetry during task execution, and compresses raw telemetry logs into concise briefs for repair. We evaluate TeleGen on WebGen-Bench and Web-Bench. On WebGen- Bench, TeleGen improves task success from 67.7% with repair without telemetry to 76.2%, an increase of 8.5 percentage points. On Web-Bench, it improves cumulative Pass@2 from 21.7% to 29.8%. Ablation results show that runtime telemetry provides a useful diagnostic signal, while telemetry briefs make this signal more effective and less costly to use. Further analysis shows that telemetry is especially helpful for failures involving hidden execution paths, such as navigation, form workflows, and frontend-backend coordination.
CODEPIVOT: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning with Reduced Reliance on Parallel Corpora
Shangyu LI | Juyong Jiang | Meibo Ren | Sizhe Zhong | Huiri Tan | Yunhao Gou | Xu Han | Chun Yong Chong | Yun Peng | Jiasi Shen
Findings of the Association for Computational Linguistics: EMNLP 2026
Shangyu LI | Juyong Jiang | Meibo Ren | Sizhe Zhong | Huiri Tan | Yunhao Gou | Xu Han | Chun Yong Chong | Yun Peng | Jiasi Shen
Findings of the Association for Computational Linguistics: EMNLP 2026
Transpilation, or code translation, aims to convert source code from one programming language (PL) to another. Recent large language model (LLM)-based approaches have demonstrated immense potential for code translation. Training-based methods are important because LLMs currently do not effectively adapt to domain-specific settings that suffer from a lack of knowledge without targeted training. However, existing training-based approaches rely on a pairwise transpilation paradigm, making it impractical to support a diverse range of PLs. Furthermore, these methods shape the reinforcement learning (RL) reward linearly in the test-case pass rate, which underweights the earliest partial progress. To address these limitations, we propose CodePivot, a training framework that leverages Python as an intermediate representation (IR), augmented by a novel RL reward mechanism, Aggressive-Partial-Functional reward, to bootstrap the model’s multilingual transpilation ability with reduced reliance on parallel corpora. Experiments involving 10 PLs show that the resulting 7B model, trained on Python-to-Others tasks, consistently improves performance across both general and low-resource PL-related transpilation tasks. It outperforms substantially larger mainstream models with hundreds of billions more parameters, such as DeepSeek-R1 and Qwen3-235B-A22B-Instruct-2507, on Python-to-Others tasks and Others-to-All tasks, respectively, and surpasses its counterpart trained directly on Any-to-Any tasks on general transpilation tasks. Code and data: https://github.com/lishangyu-hkust/CodePivot.
Learning to Code by Learning to Verify: Execution-Grounded Co-Evolution of Code Generation and Self-Verification for Reliable Code LLMs
Juyong Jiang | Chansung Park | Jiasi Shen | Sunghun Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Juyong Jiang | Chansung Park | Jiasi Shen | Sunghun Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) have made rapid progress in code generation, yet their ability to verify whether a program is correct has not necessarily improved at the same pace. We identify a generation-verification gap in code LLMs, where scaling substantially improves test-passing code generation, while execution-free correctness reasoning improves much more slowly. This gap challenges existing self-rewarding approaches, which often implicitly assume that a stronger generator also serves as a better judge. To bridge this gap, we propose CoVera, an execution-grounded co-evolution framework that jointly trains a code LLM to generate code and to verify code correctness. In generator mode, the model produces candidate programs and receives execution-based rewards from available tests. In verifier mode, the same model reads a problem-program pair, produces a reasoning trace, and predicts a correctness judgment, which is rewarded according to agreement with execution-derived labels. The calibrated verifier is then used as an auxiliary self-verification signal for generation, while the evolving generator continuously supplies increasingly challenging candidates for verifier training. This co-evolution enables an on-policy curriculum for correctness reasoning and grounds self-verification in execution feedback, without requiring exhaustive test suites or golden reference solutions. Extensive experiments across multiple benchmarks and model scales show that CoVera improves both code generation and intrinsic correctness verification, outperforming representative baselines from diverse code-learning paradigms.
ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning
Juyong Jiang | Jiasi Shen | Sunghun Kim | Kang Min Yoo | Jeonghoon Kim | Sungju Kim
Findings of the Association for Computational Linguistics: ACL 2026
Juyong Jiang | Jiasi Shen | Sunghun Kim | Kang Min Yoo | Jeonghoon Kim | Sungju Kim
Findings of the Association for Computational Linguistics: ACL 2026
While Large Language Models (LLMs) have revolutionized code generation, standard “System 1” approaches that generate solutions in a single forward pass often hit a performance ceiling on complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model’s weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-only training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B-14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, efficient reasoning and reflection patterns. The source code and data are available at https://github.com/juyongjiang/ReflexiCoder.
Do LLM-Generated Web Apps Behave Correctly? A Contract-Grounded Benchmark for Behavioral Conformance
Chansung Park | Juyong Jiang | Jiasi Shen | Sunghun Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chansung Park | Juyong Jiang | Jiasi Shen | Sunghun Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models and coding agents can produce web applications that appear functional, yet surface plausibility is an unreliable proxy for behavioral conformance under interaction. We introduce ConformWeb, a contract-grounded protocol and benchmark that operationalizes public behavioral contracts to construct hidden interaction scenarios, evaluating whether agent-generated applications preserve strict conformance under dynamic execution. By decoupling hidden interaction sequences from the public behavioral scope, the protocol guarantees a fair held-out workflow evaluation without relying on undisclosed reference implementations. Evaluated across 24 validated instances scaling to 756 evaluation scenarios and 6,433 semantic steps, only 252 of the 2,160 total execution runs achieve full conformance, yielding a strict pass rate of 11.7%. While scenario-level conformance reaches 36.7% and step-level progress averages 49.4%, this quantitative gap demonstrates that generative systems frequently achieve substantial partial progress but fail to maintain the critical behavioral invariants required for end-to-end conformance. All code and data are released at https://github.com/deep-diver/ConformWeb.