Soft latent reasoning extends chain-of-thought reasoning beyond discrete token sequences by using continuous concept tokens. However, effectively training models in this space remains challenging: conventional reinforcement learning methods mainly optimize sampled rollout-action likelihoods and therefore underuse the full distributions produced during soft rollouts. We propose **Full-Distribution Self-Distillation (FDSD)**, an off-policy training framework that treats stochastic soft rollout distributions as dense pseudo-labels. FDSD aligns the current model distribution with advantage-weighted rollout targets via KL minimization and importance correction, enabling the model to learn from the full rollout distribution rather than only from sampled-action likelihoods. Experiments on GSM8K, MATH-500, OlympiadBench, and out-of-domain benchmarks show that FDSD achieves consistent average improvements in mathematical reasoning across model scales while preserving OOD generalization, with only modest computational overhead. These results suggest that full-distribution alignment is a promising training direction for soft latent reasoning.

Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to adversarial “jailbreak” attacks designed to bypass safety guardrails. Current safety alignment methods depend heavily on static external red teaming, utilizing fixed defense prompts or pre-collected adversarial datasets. This leads to a rigid defense that overfits known patterns and fails to generalize to novel, sophisticated threats. To address this critical limitation, we propose empowering the model to be its own red teamer, capable of achieving autonomous and evolving adversarial attacks. Specifically, we introduce Safety Self- Play (SSP), a system that utilizes a single LLM to act concurrently as both the Attacker (generating jailbreaks) and the Defender (refusing harmful requests) within a unified Reinforcement Learning (RL) loop, dynamically evolving attack strategies to uncover vulnerabilities while simultaneously strengthening defense mechanisms. To ensure the Defender effectively addresses critical safety issues during the self-play, we introduce an advanced Reflective Experience Replay Mechanism, which uses an experience pool accumulated throughout the process. The mechanism employs a Upper Confidence Bound (UCB) sampling strategy to focus on failure cases with low rewards, helping the model learn from past hard mistakes while balancing exploration and exploitation. Extensive experiments demonstrate that our SSP approach autonomously evolves robust defense capabilities, significantly outperforming baselines trained on static adversarial datasets and establishing a new benchmark for proactive safety alignment.

Medical language models are increasingly evaluated on open-ended clinical reasoning, where good responses must combine factual recall, evidence integration, decision making, and synthesis. However, standard reward models compress these different standards into one reward score, while medical RL with answer keys mainly rewards multiple-choice correctness. This makes it difficult to improve non-multiple-choice medical reasoning without losing performance on exam-style benchmarks. In this paper, we propose a Prefix Mixture-of-Experts Reward Model (Prefix-MoE RM), which routes each query-response pair across recall, analysis, decision, and synthesis prefix experts, softly fuses their trainable prefixes, and then predicts one reward score. The MoE structure allows different abilities to use different scoring experts to reduce reward conflicts, while continuous prefixes provide a more expressive conditioning signal than fixed hard prompts. The reward model is trained on non-multiple-choice medical preferences and then used in mixed-data curriculum GRPO, where multiple-choice prompts provide additional answer-key feedback. On held-out reward-model evaluation, Prefix-MoE RM improves pairwise accuracy from 79.8 to 93.0 over a shared RM; with the RL pipeline fixed, replacing it with a shared RM or soft expert-head MoE lowers HealthBench from 43.8 to 24.8 and 34.6, respectively. With Qwen3-8B as the policy backbone, the full system outperforms the larger Qwen3-32B base on MedXpertQA and HealthBench, while the Qwen3-32B version reaches 50.1 HealthBench and 4.55 LLMEval-Med, remaining competitive with stronger closed general models and retaining an advantage on non-multiple-choice metrics.

Document-level relation extraction (DocRE) aims to identify relationships between entities within a document. Due to the vast number of entity pairs, fully annotating all fact triplets is challenging, resulting in datasets with numerous false negative samples. Recently, self-training-based methods have been introduced to address this issue. However, these methods are purely black-box and sub-symbolic, making them difficult to interpret and prone to overlooking symbolic interdependencies between relations.To remedy this deficiency, our insight is that symbolic knowledge, such as logical rules, can be used as diagnostic tools to identify conflicts between pseudo-labels. By resolving these conflicts through logical diagnoses, we can correct erroneous pseudo-labels, thus enhancing the training of neural models.To achieve this, we propose LogicST, a neural-logic self-training framework that iteratively resolves conflicts and constructs the minimal diagnostic set for updating models. Extensive experiments demonstrate that LogicST significantly improves performance and outperforms previous state-of-the-art methods. For instance, LogicST achieves an increase of 7.94% in F1 score compared to CAST (Tan et al., 2023a) on the DocRED benchmark (Yao et al., 2019). Additionally, LogicST is more time-efficient than its self-training counterparts, requiring only 10% of the training time of CAST.