This paper focuses on automatically generating informative ad descriptions in sponsored search. Unlike ad titles which are usually optimized to attract user click feedbacks, ad descriptions have a longer text span and possess the potential of incorporating world knowledge to address user search intents while presenting the fine-grained selling points of the ads. We propose Interactor, a multi-turn iterative creation framework optimized with agentic RL for ad description generation. The generation model acts as a policy that interacts with a customized environment consisting of multiple generative reward models. Given initial generations by the policy, the customized GenRMs evaluate qualities including knowledge capacity and landing page consistency, providing both binary signals and detailed feedbacks. The policy then iteratively refines the descriptions based on such feedbacks to ensure continuous improvement. Experiments show that it significantly outperforms state-of-the-art ad text generation approaches in generating knowledge-rich and faithful ad descriptions. Since late May 2026, it has been deployed online in a leading search ads system, where the framework serves over 140k advertisers, contributing to both ad revenue and user experience.

An advertising platform typically generates a diverse candidate set of text creatives for each ad and then employs personalized online ranking to estimate the top-performing one for final display. Conventionally, the diversity is treated as a by-product during the stochastic decoding process rather than a training objective, which results in suboptimal outputs due to the lack of explicit modeling for semantic coverage. In this paper we reformulate ad text generation as a set-level modeling problem, and propose a parallel generation framework that optimizes the diversity objective with set-level alignment. By explicitly decomposing an ad’s content structure into semantic-independent perspectives via a planning mechanism, our framework generates the entire set of text creatives in parallel based on the planned semantics, which also takes full advantage of parallelizability to enable high-throughput industrial deployment. It is optimized through a set-level alignment stage that encourages the overall semantic coverage and penalizes the redundancy among the candidates. Comprehensive experiments on benchmark dataset show that our framework achieves better set-level quality and diversity, as well as significantly improving generation throughput, demonstrating its superiority on both generation quality and inference speed.

Continuous diffusion language models struggle with structured generation under hard, variable-length templates. We identify two failure modes: Boundary Attention Drift, in which head-region attention bleeds into padded tails under block-mode training, and Vocab Projection Drift, in which latent-to-token rounding over-projects to low-frequency tokens. The two modes dissociate cleanly across Chinese ci, English sonnets, and E2E NLG: the former is specific to block-mode training, whereas the latter appears in all three datasets regardless of padding regime, isolating padding regime and rounding rule as orthogonal sources of error. To address structural correctness, we adapt constrained decoding to diffusion rounding through a position-template primitive, the V_i rule, instantiated on continuous-diffusion logits as Length-Conditional Voronoi Rounding (LCVR). The same primitive applied to a small autoregressive baseline reaches identical zero-violation rates, showing that structural correctness is delivered by the template, not by the underlying model family. A contribution more readily realised with diffusion backbones is instead cross-domain backbone transfer: initialising a target-domain model from a structurally compatible source-domain backbone yields order-of-magnitude reductions in training steps, while a controlled negative-transfer arm fails to reproduce the speed-up on structural metrics; an attention-span probe localises the transferable component to the source backbone’s long-range attention span rather than to its padding regime or corpus size. We further report a calibrated wallclock analysis showing that the dominant cost of diffusion inference is recoverable with standard step-reduction techniques. Together, these results recast several previously scattered structured-generation failures of continuous diffusion as one consistent, transferable, and fixable phenomenon.

Group-relative policy optimization (GRPO, DAPO, ARPO) has become the dominant recipe for RL fine-tuning of reasoning LLMs, yet in the resource-constrained regime most practitioners train in (a single GPU, small batches) it consistently suffers from rapid entropy collapse that kills exploration, together with sparse credit assignment in long rollouts, which is most extreme under multi-turn tool use but already present in long single-turn chain-of-thought. We address both failure modes with JANUSPO, a drop-in GRPO modification that introduces asymmetric PPO clipping 𝜖₊<𝜖_- to tolerate larger downward than upward updates, pairs it with an asymmetric KL schedule 𝛽_pos<𝛽_neg that doubles as a free entropy regulator without any explicit upper bound, and redistributes the terminal reward through a turn-weighted decomposition w_t(n) that generalises naturally from single-turn CoT to multi-turn rollouts. On Qwen2.5-3B-Instruct / GSM8K, JANUSPO improves from 74.8% to 84.3±0.2% (mean±std over 2 seeds) across 14 controlled ablations, and the same recipe lifts 7B from 85.3% to 93.6% with no hyper-parameter retuning. On multi-turn DeepSearch (103 GAIA + 100 HLE), SFT-warmup-then-JANUSPO is the only RL recipe whose paired comparison against SFT survives best@8 outlier verification, achieving strict dominance on all four reward and F1 columns while a matched-init GRPO baseline’s apparent F1 wins collapse to 0%. Beyond this main result, we document a systematic negative finding: SFT warm-up is a necessary precondition for GRPO-style multi-turn RL on format-unaware base models.

Chain-of-thought Distillation (CoTD) aims at distilling Chain-of-thought (CoT) reasoning ability of large language models (LLMs) to much smaller student models. The core of CoTD is using a large teacher model to generate rationales and fine-tune smaller student models. However, current Chain-of-thought Distillation works have the following limitations: 1) Student models are separately distilled from specific reasoning tasks and lack a collaboration mechanism, hindering the enhancement of reasoning performance through collaboration among various reasoning tasks. 2) The parameter update of student models severely harms the CoT reasoning ability on other unseen reasoning tasks not included in the distillation process. In this work, we introduce a novel CoT Distillation method, MoDE-CoTD, which decouples the CoT reasoning abilities out of the student model by distilling multiple LoRA-Experts and freezing the parameters of the student model. Sequentially, LoRA-Experts are combined and adapted to handle both seen and unseen reasoning tasks, enabling collaboration among diverse reasoning tasks to further enhance CoT reasoning performance. Experimental results on 14 datasets (including 4 unseen datasets) demonstrate the strength of MoDE-CoTD, with an average accuracy gain of 6.3% on seen datasets and 7.8% on unseen datasets.

Aligning large language models (LLMs) with human expectations requires high-quality instructional dialogues, which can be achieved by raising diverse, in-depth, and insightful instructions that deepen interactions. Existing methods target instructions from real instruction dialogues as a learning goal and fine-tune a user simulator for posing instructions. However, the user simulator struggles to implicitly model complex dialogue flows and pose high-quality instructions. In this paper, we take inspiration from the cognitive abilities inherent in human learning and propose the explicit modeling of complex dialogue flows through instructional strategy reuse. Specifically, we first induce high-level strategies from various real instruction dialogues. These strategies are applied to new dialogue scenarios deductively, where the instructional strategies facilitate high-quality instructions. Experimental results show that our method can generate diverse, in-depth, and insightful instructions for a given dialogue history. The constructed multi-turn instructional dialogues can outperform competitive baselines on the downstream chat model.