2026

Paper-to-code reproduction asks scientific AI agents to turn research papers into executable repositories that preserve the paper’s method, protocol and artifacts. This is difficult because the specification is split: explicit paper content such as algorithms, metrics and artifacts is often lost across long agent trajectories, while implicit details such as framework defaults and conventions inherited from related work are absent from the paper. We introduce ReproAgent a four-stage Prepare–Plan–Generate–Repair pipeline built around a persistent implementation contract with two channels: an implementation-requirement channel that turns paper snippets into code obligations, and a reference-evidence channel that retrieves content and structure evidence from related repositories. Both are bound to work packages, projected into file-level contracts, and consumed across generation and repair. On PaperBench Code-Dev, ReproAgent reaches the highest mean score among same-backbone scaffolds under both Claude-Sonnet-4.5 and Gemini-3-Flash. End-to-end channel ablations and per-paper cases support the contribution of both channels.Code and experimental artifacts are publicly available at https://anonymous.4open.science/r/ReproAgent-E760.
LLM agents can generate paper reproduction code, yet often produce scientifically unfaithful implementations. We define this failure mode as semantic drift, where generated code silently diverges from the paper’s specifications. We introduce SemanticAlign-Bench(SA-Bench), a diagnostic benchmark covering 30 papers from ICLR, ICML and NeurIPS 2025. For each paper, we decompose its specifications into atomic and verifiable implementation claims, which we call Semantic Alignment Units (SAUs) and evaluate repositories along four diagnostic dimensions spanning numerical, methodological, protocol and ordering drift. In total, we construct 1,491 SAUs across five ML domains and evaluate 12 generator configurations (4 models 3 scaffolds). Even the strongest configuration (Claude + PaperCoder) achieves a mean SAU score of only 0.301 out of 1.0, with an overall mean of 0.221 across 360 evaluations. A failure taxonomy reveals that agents attempt most requirements but implement them incorrectly, with implementation mismatch and stubs accounting for the majority of zero-scored claims. Our analysis further indicates that scaffolds optimized for executability provide limited leverage for scientific reproduction; narrowing the gap requires scaffolds that prioritize semantic specification verification. The benchmark, annotations and evaluation pipeline are publicly available.

Code:https://anonymous.4open.science/r/SemanticAlign-Bench-0676

Data:https://anonymous-hf.up.railway.app/a/rrgn430zpfui/
Real-world analysis often requires combining directly queryable structured sources (e.g., SQL and CSV) with evidence preserved only in visual documents, such as scanned reports, invoices, and dashboards. Current analytics agents still focus primarily on structured and textual inputs, leaving this cross-modal setting underexplored. We introduce DataCross, a benchmark and agent framework for heterogeneous, insight-driven data analysis. DataCrossBench contains 200 end-to-end tasks across six fine-grained categories: Fund Analysis, FinTech, Internet Development, Internet Firms, Short-form Drama, and AI Healthcare, grouped into four macro sectors and mixing SQL/CSV/JSON/TXT sources with image-based evidence. The benchmark is built with a human-in-the-loop reverse-synthesis pipeline to preserve cross-source dependency while keeping the target insights verifiable. We further propose DataCrossAgent, which coordinates source-specialized agents through intra-source exploration, source prioritization, cross-source pollination, and Reasoning ReAct (reReAct)-based executable verification. Experiments show that DataCrossBench reveals a difficulty gap for most baselines, especially on vision-dependent Hard tasks, and that DataCrossAgent improves average factuality over vanilla GPT-4o from 0.3291 to 0.4270 (+29.7%) while showing a smaller Easy-to-Hard degradation than several baselines. These results position DataCrossBench as a practical testbed for cross-modal heterogeneous data analysis.
Low-resource target-language generation is often limited by scarce parallel data, while high-resource source-language monolingual data is abundant but difficult to use with standard supervised fine-tuning. We propose Source-Grounded Semantic Reinforcement Learning (SG-SRL), a resource-utilization framework that converts source-language monolingual data into cross-lingual semantic supervision. SG-SRL performs reference-free reinforcement learning (RL) on source-language data with a cross-lingual semantic reward, followed by lightweight supervised recovery. Although direct reward optimization induces verbosity-based reward hacking, recovery restores fluency, conciseness, and task format without erasing the learned semantic alignment. On Chinese-to-Thai generation, SG-SRL outperforms cold-start SFT, source-only continued pretraining, and synthetic-data SFT; human preference and training-independent metrics corroborate the gains. Recovery ablations further show that semantic retention and form restoration are jointly controlled by recovery data and intensity. An in-domain Chinese-to-Tibetan experiment shows that the reward interface can use a contrastively trained encoder when a capable multilingual reranker is unavailable, while also delimiting the framework’s dependence on reward-model discrimination.