Adaptive context compression is vital for scaling Large Language Models (LLMs) to complex, multi-turn agent tasks. However, rule-based compression methods may discard task-critical nuances, while Reinforcement Learning (RL) approaches usually struggle to balance information retention and token efficiency under the sparse rewards inherent to long-horizon workflows. To bridge this gap, we propose ZipRL, a novel adaptive compression framework tailored for Reinforcement Learning from Verifiable Rewards (RLVR). ZipRL features a multi-granularity compression mechanism for active, non-uniform information reduction, coupled with Hindsight Response Replay (HRR)—a technique designed to densify training signals during RLVR optimization. Theoretically, we prove ZipRL’s superior task-relevant utility over uniform methods. Concretely, ZipRL utilizes coarse-to-fine prompts for macro-compression and incorporates HRR into GRPO via generalized advantage reshaping. Multiple models of varying versions and parameter scales validate the effectiveness of our approach. Benchmarks on five agent tasks show ZipRL outperforms state-of-the-art approaches by 27.9% and 34.7% across Qwen3-4B and Qwen3-8B models, while maintaining exceptional token efficiency and robustness under extreme 256-turn extrapolation stress tests. Our code is available at https://github.com/huzhexin/ZipRL.

Training LLMs for tool use requires more than final-answer supervision: models need trajectories that specify how to decompose a question, call tools across turns, use retrieved evidence, and recover from earlier errors. Existing data construction pipelines often rely on large-scale real-world tool API calls or simple templates, making them costly or limited in interaction coverage. We introduce ToolForge, a virtual-tool data synthesis pipeline for constructing tool-use datasets with three targeted abilities: multi-hop reasoning, multi-turn tool use, and self-reflection. Starting from QA seed data, ToolForge builds an extensible tool space from 19 base virtual tools and automatically generated variants, selects among four tool-use paradigms, simulates both successful and misleading retrieval outcomes, and generates validated tool-use dialogues without large-scale real-world tool API execution. A Multi-Layer Validation module filters invalid traces and is audited with hard negatives mined by Monte Carlo Tree Search. Fine-tuning Qwen3-8B on only 4,250 ToolForge samples produces a model named ToolForge-8B, which improves over the Tool-Integrated Reasoning baseline by 21.6% on five out-of-domain QA benchmarks and remains competitive on BFCL and API-Bank. We further quantify the pipeline construction cost, including tool initialization, dialogue generation, validation, and SFT, and compare ToolForge with existing synthetic tool-data sources. Our code and dataset are publicly available at https://github.com/Buycar-arb/ToolForge.