We present the setup, systems, evaluation, and findings of the WMT26 Video Subtitle Translation shared task. The task covers translation from Simplified Chinese into English, Thai, Indonesian, Malay, and Traditional Chinese for Taiwan, China. Eight teams submitted 2,200 SRT files across 22 team-language combinations. We evaluated 2,198 unique team-language-video units with repeated LLM-based language quality assessment followed by human checking and correction. Each video was assessed three times, with valid positive scores aggregated by the median. ReopenAI led all five directions and achieved a macro-average of 89.357; laniqo ranked second with 78.265. Corrected scores were extractable from 93.36\% of the evaluation units, and 15.30\% of those scores differed from the automatic scores. Stronger results were associated with contextual prompting, diverse candidate generation, constrained selection, translator-critic specialization, terminology control, and deterministic subtitle validation. These cross-system associations are not controlled ablation evidence. Persistent challenges include cross-cue reference resolution, cultural and register adaptation, multilingual judge calibration, and balancing brevity with semantic completeness.

General-purpose machine translation benchmarks such as FLORES-200 have reached a saturation regime on Chinese-English pairs, where modern large language models cluster within a narrow band of high scores. Across 22 systems, FLORES-200 zh-en GEMBA scores fall in a 7.87-point range with a standard deviation of 2.29, which compresses the separation between systems on knowledge-intensive domains such as finance, healthcare, law, and science and technology. We introduce HardMTBench, a difficulty-aware diagnostic benchmark for bidirectional Chinese-English domain translation. HardMTBench covers 12 domains and contains 10000 hand-curated source sentences with reference translations, packaged as 20000 directional test items. A three-stage construction pipeline builds a domain-balanced candidate pool of 84566 pairs, applies an LLM-based multi-signal judge over knowledge density, translation difficulty, terminology load and reference correctness, and assembles the final test set under a hardness fusion rule with per-domain quotas. Across 22 systems spanning general LLMs, commercial engines and specialised MT models, HardMTBench widens the cross-system GEMBA range by roughly a factor of two over FLORES-200, induces visible rank reorderings, and exposes domain-specific terminology and knowledge weaknesses that quality-only metrics tend to flatten.

Long-term memory for an LLM agent requires more than retrieving a relevant passage. Many existing systems address belief revision, causal connections and cross-domain abstraction primarily through storage and retrieval mechanisms, limiting their ability to support implicit personalisation as a user’s preferences evolve. We present DPCM, which organises memory into a capability hierarchy spanning raw inputs, atomic facts, belief trajectories, identity, domain schemas, prospective intentions and cross-domain patterns. A synchronous daytime writer (SYSTEM 1) records revisions as doubly linked supersedes chains; an asynchronous nighttime engine (SYSTEM 2) induces schemas and intentions, then uses cross-domain collisions to derive core schemas. Dual-process theory motivates this separation between online encoding and offline abstraction without implying that agent memory reproduces human cognition. Across single-run evaluations on LongMemEval, PersonaMem and PersonaMem-v2, SYSTEM 2 yields its largest observed gain on implicit cross-session inference (+5.20 on PersonaMem-v2) and its smallest on the recall-oriented LongMemEval aggregate (+0.33). When the inference budget is limited to 8k tokens and the full history no longer fits, DPCMfull exceeds the long-context baseline by 10.5–17.9 points across both backbones.

Modern translation workflows demand more than semantic equivalence. Users routinely require models to preserve JSON or HTML schemas, honor curated glossaries, disambiguate with provided context, and match prescribed registers, often several at once. Conventional metrics such as BLEU and xCOMET capture semantic fidelity but provide little signal on constraint adherence, while general instruction following benchmarks ignore the cross-lingual nature of translation. We introduce IFMTBench, a benchmark for multilingual translation instruction following covering seven high-resource languages, with 4,506 single-constraint and 2,838 multi-constraint items spanning seven constraint dimensions and five compositional patterns with instructions issued in all seven languages. Constraints are split into a gating subset verified by deterministic checkers and a continuous subset scored by a rubric-based LLM judge, combined under a multiplicative rule that resists reward hacking. Evaluating 15 models reveals systematic gaps that prior protocols miss: Instruction following scales with size more sharply than translation quality, glossary and structured-format constraints dominate the difficulty gradient, and general instruction following rankings correlate only weakly with translation behavior. Our benchmark is available at https://github.com/Tencent-Hunyuan/Hy-MT2/tree/main/IFMTBench.

Large language models (LLMs) have recently demonstrated remarkable capabilities in machine translation (MT). However, most advanced MT-specific LLMs rely heavily on external supervision during training, such as human-annotated reference data or trained reward models (RMs), which are expensive to obtain and difficult to scale. To address this limitation, we propose Simple Self-Rewarding (SSR), a reinforcement learning (RL) framework for MT that is reference-free and relies solely on self-judging rewards. Using only 13K monolingual examples and Qwen-2.5-7B as the backbone, SSR-Zero-7B outperforms existing MT-specific LLMs as well as larger general LLMs such as Qwen2.5-32B-Instruct on English ↔ Chinese translation benchmarks including WMT23, WMT24, and FLORES200. It further demonstrates strong generalization to low-resource language pairs. In addition, when augmented with external supervision from COMET, our strongest model, SSR-X-Zero-7B, surpasses all existing open-source models under 72B parameters and performs competitively with leading closed-source systems in English ↔ Chinese translation. Our analysis highlights the effectiveness and generalizability of the self-rewarding mechanism relative to external LLM-as-a-judge approaches and demonstrates its complementary benefits when combined with trained RMs. We will publicly release our code, data, and models.

Large Language Models (LLMs) have advanced machine translation, yet long-text translation still faces a practical tension between document-level lexical consistency and inference cost. Recent document-level methods improve coherence by expanding context, maintaining multi-level memories, or injecting multi-granularity knowledge, but such broad context mechanisms often introduce substantial prompt redundancy and limit parallel inference. We propose NoBAT (Noun Bank Augmented Translation), a lightweight framework that distills document-level lexical constraints into a compact global Noun Bank. NoBAT first constructs and disambiguates the Noun Bank serially, and then retrieves only relevant noun constraints for local, parallel translation. Experiments on IWSLT2017 and Novels-XL with multiple LLM backbones show that NoBAT consistently improves lexical translation consistency while maintaining competitive COMET scores. Compared with DELTA, NoBAT further reduces total token consumption by 6.3×, demonstrating a cost-effective trade-off among consistency, translation quality, and efficiency.

Agreement among LLM judges is widely used as evidence that automated evaluation is reliable. Yet consensus may be stable without reflecting a shared understanding of quality. We call this Evaluation Illusion: judges can produce detailed critiques while relying on overlapping surface cues or a common evaluation instrument. In a fully crossed baseline study, three frontier judges conduct 105,600 evaluations of 32 models. First, for the two highest-scoring models, moving from a shared five-dimension rubric package to independently generated rubrics reduces agreement by 62%, a joint contrast we term the Rubric Commensurability Problem. Second, nearly identical model-level rankings coexist with substantially weaker criterion-level agreement under non-equivalent metrics and sample sets, yielding the Resolution Paradox. We further introduce MERG, a four-stage protocol that elicits task knowledge and potential biases before scoring. In a controlled comparison on the same models at t=0.0, agreement decreases under independently generated MERG rubrics. These results do not establish that MERG scores are more accurate; they show that consensus depends on the evaluation instrument and analytical resolution at which it is measured.

Multi-turn agents that plan, invoke tools, and interact with environments offer a promising paradigm for solving complex tasks, yet their capabilities typically rely on very large models whose inference cost is prohibitive in practice. On-Policy Distillation (OPD) is a natural recipe for transferring such capabilities to smaller students, but we find that it suffers a characteristic failure mode in this setting: small student errors compound across turns and push the trajectory out of the teacher’s familiar state distribution, so the teacher’s supervision becomes least reliable precisely where the student needs it most. We propose Guided On-Policy Distillation (Guided-OPD), a simple yet effective algorithm that mixes teacher- and student-generated turns within each rollout and schedules the teacher’s intervention probability along a curriculum that decays to zero. Strong guidance keeps early trajectories close to the teacher distribution and is then gradually withdrawn to recover the purely on-policy regime used at inference. On ALFWorld, ScienceWorld, and WebShop, distilling Qwen3 students from a Qwen3-30B-A3B teacher, Guided-OPD yields average relative gains of 21.1% in Score and 25.5% in Success Rate over vanilla OPD, with larger gains on smaller students.

Podcast script generation requires LLMs to synthesize structured, context-grounded dialogue from diverse inputs, yet systematic evaluation resources for this task remain limited. To bridge this gap, we introduce PodBench, a benchmark comprising 800 samples with inputs up to 21K tokens and complex multi-speaker instructions. We propose a multifaceted evaluation framework that integrates quantitative constraints with LLM-based quality assessment. Extensive experiments reveal that while proprietary models generally excel, open-source models equipped with explicit reasoning demonstrate superior robustness in handling long contexts and multi-speaker coordination compared to standard baselines. However, our analysis uncovers a persistent divergence where high instruction following does not guarantee high content substance. PodBench offers a reproducible testbed to address these challenges in long-form, audio-centric script generation.

In this paper, we present our submission to the WMT25 shared task on machine translation, for which we propose Synergy-enhanced policy optimization framework, named Shy. This novel two-phase training framework synergistically combines knowledge distillation and fusion via reinforcement learning.In the first phase, we introduce a multi-stage training framework that harnesses the complementary strengths of multiple state-of-the-art large language models to generate diverse, high-quality translation candidates. These candidates serve as pseudo-references to guide the supervised fine-tuning of our model, Hunyuan-7B, effectively distilling the collective knowledge of multiple expert systems into a single efficient model.In the second phase, we further refine the distilled model through Group Relative Policy Optimization, a reinforcement learning technique that employs a composite reward function. By calculating reward from multiple perspectives, our model ensures better alignment with human preferences and evaluation metrics.Extensive experiments across multiple language pairs demonstrate that our model Shy-hunyuan-MT yields substantial improvements in translation quality compared to baseline approaches. Notably, our framework achieves competitive performance comparable to that of state-of-the-art systems while maintaining computational efficiency through knowledge distillation and strategic ensemble.

Improving training efficiency continues to be one of the primary challenges in large-scale Reinforcement Learning (RL). In this paper, we investigate how context length and the complexity of training data influence the RL scaling training process of R1-distilled reasoning models, e.g., DeepSeek-R1-Distill-Qwen-1.5B. Our experimental results reveal that: (1) simply controlling the context length and selecting the training data based on the input prompt length can effectively improve the training efficiency of RL scaling, achieving better performance with more concise CoT; (2) properly scaling the context length helps mitigate entropy collapse; and (3) carefully choosing the context length facilitates achieving efficient LLM training and reasoning. Inspired by these insights, we propose FastCuRL, a curriculum RL framework with stage-wise context scaling to achieve efficient LLM training and reasoning. Extensive experimental results demonstrate that FastCuRL-1.5B-V3 significantly outperforms state-of-the-art reasoning models on five competition-level benchmarks and achieves 49.6% accuracy on AIME 2024. Furthermore, FastCuRL-1.5B-Preview surpasses DeepScaleR-1.5B-Preview on five benchmarks while only using a single node with 8 GPUs and a total of 50% of training steps.

Utilizing Large Language Models (LLMs) as evaluators to assess the performance of other LLMs has garnered attention. However, this evaluation approach is affected by potential biases within LLMs, raising concerns about the accuracy and reliability of the evaluation results of LLMs. To address this issue, we propose and explore two many-shot In-Context Learning (ICL) prompt templates to help LLM evaluators mitigate potential biases: Many-Shot with Reference (MSwR) and Many-Shot without Reference (MSoR). Specifically, the former utilizes in-context examples with model-generated rationales as references, while the latter does not include these references. Using these prompt designs, we investigate the impact of increasing the number of in-context examples on the consistency and quality of the evaluation results. Experimental results show that advanced LLMs, such as GPT-4, perform better in the many-shot regime than in the zero-shot regime. Furthermore, in most cases, MSwR performs significantly better than MSoR.

Despite recent efforts to develop large language models with robust long-context capabilities, the lack of long-context benchmarks means that relatively little is known about their performance. To alleviate this gap, in this paper, we propose Counting-Stars, a multi-evidence, position-aware, and scalable benchmark designed to evaluate the multi-evidence retrieval capabilities of long-context LLMs. Counting-Stars comprises two counting-based multiple pieces of evidence retrieval tasks: searching and reasoning. Using Counting-Stars, we conducted experiments to evaluate several long-context LLMs, including GPT-4 Turbo, Gemini 1.5 Pro, Claude3 Opus, GLM-4, and Moonshot-v1. Extensive experimental results demonstrate that Gemini 1.5 Pro achieves the best overall results, while GPT-4 Turbo exhibits the most stable performance across various tasks. Furthermore, our analysis of these LLMs, which have been extended to handle long-context scenarios, indicates that significant room for improvement remains as the length of the input context and the complexity of the tasks increase.

Extensive research has been conducted to explore the capability of Large Language Models (LLMs) for table reasoning and has significantly improved the performance on existing benchmarks. However, tables and user questions in real-world applications are more complex and diverse, presenting an unignorable gap compared to the existing benchmarks. To fill the gap, we propose a Multi-scale spreadsheet benchmark with Meta operations for Table reasoning, named as MiMoTable. Specifically, MiMoTable incorporates two key features. First, the tables in MiMoTable are all spreadsheets used in real-world scenarios, which cover seven domains and contain different types. Second, we define a new criterion with six categories of meta operations for measuring the difficulty of each question in MiMoTable, simultaneously as a new perspective for measuring the difficulty of the existing benchmarks. Experimental results show that Claude-3.5-Sonnet achieves the best performance with 77.4% accuracy, indicating that there is still significant room to improve for LLMs on MiMoTable. Furthermore, we grade the difficulty of existing benchmarks according to our new criteria. Experiments have shown that the performance of LLMs decreases as the difficulty of benchmarks increases, thereby proving the effectiveness of our proposed new criterion.