Weifeng Lv

Author directory

2026

While large language models (LLMs) have mastered syntax-level code generation, complex algorithmic reasoning remains a challenge, typically addressed by scaling model depth and parameter count. Universal Transformers (UT) offer a compelling alternative by introducing a recurrent inductive bias that aligns with the recursive nature of programming logic. However, training looped architectures at scale has historically been hindered by severe instability and optimization difficulties associated with backpropagation through time (BPTT). We present LoopCoder (40B-A80B) pre-trained on 12T+ code and general tokens, along with LoopCoder-Thinking and LoopCoder-Instruct variants—the first large-scale looped transformer for code, achieving comparable performance to standard dense architectures with more parameters. Unlike prior approaches that restrict recurrence to small-scale tasks, we implement a comprehensive looped training protocol spanning both pre-training and post-training phases. We initiate the model via dense-to-loop transformation, folding a pre-trained dense checkpoint to initialize a recurrent block, followed by rigorous looped pre-training and specialized post-training for instruction following and reasoning. Our results establish a robust recipe for scaling coding intelligence via recurrent computation, proving that dense checkpoints serve as an optimal foundation for evolving into dynamic, looped reasoners.
Large language models (LLMs) are powerful but costly to train, with scaling laws predicting performance from model size, data, and compute. However, different programming languages (PLs) have varying impacts during pre-training that significantly affect base model performance, leading to inaccurate performance prediction. Existing works focus on language-agnostic settings, neglecting the inherently multilingual nature of modern software development. Therefore, it is first necessary to investigate the scaling laws of different PLs, and then consider their mutual influences to arrive at the final multilingual scaling law. In this paper, we present the first systematic exploration of scaling laws for multilingual code pre-training, conducting over 1000+ experiments (Equivalent to 336,000+ H800 hours) across multiple PLs, model sizes (0.2B to 14B parameters), and dataset sizes (1T tokens). We establish scaling laws for code LLMs across multiple programming languages, showing that interpreted languages benefit more from increased scale than compiled ones. Multilingual pre-training provides synergistic benefits, especially between syntactically similar languages, with parallel pairing (concatenating code with translations) significantly enhancing cross-lingual abilities. We propose a proportion-dependent multilingual scaling law that optimally allocates training tokens by prioritizing high-utility languages (e.g., Python), balancing high-synergy pairs (e.g., JavaScript-TypeScript), and reducing allocation to fast-saturating languages (e.g., Rust), achieving superior performance across all languages compared to uniform distribution.
Geo-localization aims to infer the geographic origin of a given signal. In computer vision, geo-localization has served as a demanding benchmark for compositional reasoning and is relevant to public safety. In contrast, progress on audio geo-localization has been constrained by the lack of high-quality audio-location pairs. To address this gap, we introduce AGL1K, the first audio geo-localization benchmark for audio language models (ALMs), spanning 72 countries and territories. To extract reliably localizable samples from a crowd-sourced platform, we propose the Audio Localizability metric that quantifies the informativeness of each recording, yielding 1,444 curated audio clips. Evaluations on 16 ALMs show that ALMs have emerged with audio geo-localization capability. We find that closed-source models substantially outperform open-source models, and that linguistic clues often dominate as a scaffold for prediction. We further analyze ALMs’ reasoning traces, regional bias, error causes, and the interpretability of the localizability metric. Overall, AGL1K establishes a benchmark for audio geo-localization and may advance ALMs with better geospatial reasoning capability.
Remote sensing has emerged as an important domain for multimodal large language models (MLLMs), where reliable understanding requires locating small, dense, and spatially subtle visual evidence. However, our diagnostic analysis shows that current MLLMs often fail because their reasoning is not grounded to the correct visual evidence, leading to wrong-region attention or missed task-relevant objects. To address this issue, we introduce RsEvi-8B, an evidence-aware remote sensing MLLM that reasons with parseable bounding-box-grounded evidence. To enable this capability, we construct RsEvi-248K, a large-scale supervised dataset spanning VQA, visual grounding, object detection, object counting, and scene classification. The dataset provides verified bounding-box-grounded reasoning traces, where each intermediate visual claim is aligned with supporting evidence regions and filtered for both answer correctness and evidence faithfulness. To improve evidence-faithful reasoning, we further propose the evidence-aware GRPO, which uses an evidence verifier to regulate learning on generated evidence spans while preserving task-level outcome optimization. Extensive experiments across five remote-sensing task families show that RsEvi-8B improves the dataset-balanced average score by 17.15 points over Qwen3-VL-8B-Instruct, establishing explicit visual grounding as a key ingredient for reliable remote-sensing reasoning.
Industrial code tasks, including chip design, GPU optimization, and firmware, often fail for reasons that are invisible in ordinary source code: timing constraints, backend logs, layout data, simulator behavior, and hardware-specific performance trade-offs. Current code models see many programs but few expert correction traces from these settings. We present InCoder-32B-Thinking, a 32B thinking code model trained with Error-driven Chain-of-Thought (ECoT) synthesis and an Industrial Code World Model (ICWM). ECoT converts several rounds of toolchain signals into reasoning traces by preserving the failed code, the observed failure, and the later fix. ICWM learns from execution traces in domains such as Verilog simulation and GPU profiling, then predicts feedback for the data engine, which can build more repair traces without routing every candidate through a real toolchain. The data keeps the uneven reasoning depth of industrial work: short traces for simple cases, long traces for hardware-level debugging. Across 23 tests (14 general and 9 industrial), InCoder-32B-Thinking reaches 81.3% on LiveCodeBench v5, 84.0% CAD-Coder compile success, and 38.0% on KernelBench L2, while remaining competitive on general coding tasks. Model weights and code are available at Hugging Face: https://huggingface.co/Multilingual-Multimodal-NLP/IndustrialCoder and GitHub: https://github.com/CSJianYang/Industrial-Coder.

2017

We present a simple yet effective approach for linking entities in queries. The key idea is to search sentences similar to a query from Wikipedia articles and directly use the human-annotated entities in the similar sentences as candidate entities for the query. Then, we employ a rich set of features, such as link-probability, context-matching, word embeddings, and relatedness among candidate entities as well as their related entities, to rank the candidates under a regression based framework. The advantages of our approach lie in two aspects, which contribute to the ranking process and final linking result. First, it can greatly reduce the number of candidate entities by filtering out irrelevant entities with the words in the query. Second, we can obtain the query sensitive prior probability in addition to the static link-probability derived from all Wikipedia articles. We conduct experiments on two benchmark datasets on entity linking for queries, namely the ERD14 dataset and the GERDAQ dataset. Experimental results show that our method outperforms state-of-the-art systems and yields 75.0% in F1 on the ERD14 dataset and 56.9% on the GERDAQ dataset.

2016