State-of-the-art (SOTA) reasoning post-training pipelines increasingly combine supervised fine-tuning (SFT) with reinforcement learning (RL) to balance external knowledge injection and self-exploration. In this work, we identify a critical but underexplored failure mode of these SFT+RL pipelines: catastrophic forgetting. To analyze the cause, we reveal a fundamental asymmetry between SFT and RL. SFT typically induces much larger, redundant parameter updates than RL, while RL’s updates are smaller and more parsimonious. Therefore, SFT updates can easily overwrite RL’s small non-robust updates, causing the forgetting of the knowledge learned by RL. To address this issue, we propose MIFO (MItigating FOrgetting between SFT and RL), a plug-and-play framework that integrates SFT into RL while explicitly constraining SFT-induced updates. MIFO mitigates forgetting through two complementary designs: (i) Data selection, which interleaves RL and SFT by selecting challenging samples based on rollout accuracy and optimizing only high-entropy tokens for SFT; and (ii) parameter freezing, which identifies RL-critical parameters and temporarily freezes them during SFT to prevent overwrite. Beyond stability, MIFO is data- and response-efficient and remains agnostic to the underlying SFT or RL algorithm. Empirically, MIFO achieves SOTA reasoning performance using only 1.5% of the SFT data and 20.4% of the RL data required by prior SOTA methods, while producing reasoning traces up to 10× shorter than the standard SFT→RL pipeline.

Language model (LM) agents are increasingly deployed to maintain codebases over extended periods, fixing streams of related defects while carrying context from one fix to the next. Yet existing software engineering (SWE) benchmarks evaluate models one bug at a time: the repository is reset, the codebase is re-read, and a single self-contained issue is graded in isolation. This setting collapses a continuous maintenance workflow into a series of independent sessions, ignoring the cumulative dependencies that make real-world bug fixing challenging. To bridge this gap, we introduce ChainSWE, the first benchmark for evaluating agents on sequential, dependent bug fixes within a shared codebase. We collect chronological chains of 304 issues across 54 Python projects, mined from six SWE-bench-family datasets. Our evaluation across a range of agents and models reveals a consistent performance drop by **up to 70%** as the chain length increases. Our data and code is available at [C-lister/ChainSWE](https://huggingface.co/datasets/C-lister/ChainSWE).

Reinforcement learning (RL) is central to post-training, particularly for agentic models that require specialized reasoning behaviors. In this setting, model merging offers a practical mechanism for integrating multiple RL-trained agents from different tasks into a single generalist model. However, existing merging methods are designed for supervised fine-tuning (SFT), and they are suboptimal to preserve task-specific capabilities on RL-trained agentic models. The root is a task-vector mismatch between RL and SFT: on-policy RL induces task vectors that are highly sparse and heterogeneous, whereas SFT-style merging implicitly assumes dense and globally comparable task vectors. When standard global averaging is applied under this mismatch, RL’s non-overlapping task vectors that encode critical task-specific behaviors are reduced and parameter updates are diluted. To address this issue, we propose Reinforced Agent Merging (RAM), a distribution-aware merging framework explicitly designed for RL-trained agentic models. RAM disentangles shared and task-specific unique parameter updates, averaging shared components while selectively preserving and rescaling unique ones to counteract parameter update dilution. Experiments across multiple agent domains and model architectures demonstrate that RAM not only surpasses merging baselines, but also unlocks synergistic potential among agents to achieve performance superior to that of specialized agents in their domains.

Large Language Models (LLMs) rely heavily on large-scale reasoning data, but as such data becomes increasingly scarce, model self-improvement offers a promising alternative. However, this process can lead to model collapse, as the model’s output becomes overly deterministic with reduced diversity. In this work, we identify a new risk beyond model collapse, which we term the Superficial Self-Improved Reasoners phenomenon. This phenomenon indicates that while self-improvement enhances in-domain (ID) reasoning accuracy, it degrades the model’s generalized reasoning capability on out-of-domain (OOD) datasets, as the model tends to memorize the training data. Our analyses of layer importance and parameter changes reveal that reasoning-critical layers receive fewer updates compared to less relevant layers during self-improvement. To address this, we propose Iterative Model Merging (IMM), which balances reasoning improvements and generalization by merging the weights of the original and self-improved models. IMM effectively mitigates model collapse and improves generalized reasoning capability. Code is available at https://github.com/xiangchi-yuan/merge_syn