Siyao Song

Author directory

2026

Reinforcement learning with outcome-based objectives such as GRPO enables LLM-based agents to solve complex, long-horizon tasks, yet the reusable exploration patterns embedded in interaction trajectories are largely discarded after a single policy update. Existing experience-augmented approaches retrieve historical guidance at inference time, but they apply experiences without accounting for the policy’s evolving capability and create persistent dependencies on external retrieval. We propose EDGE (Experience-Distillation for Guided Exploration), a framework that treats retrieved experiences as temporary training-time scaffolds and progressively internalizes their benefits into the parametric policy. Concretely, EDGE partitions each rollout group into experience-conditioned and experience-free trajectories to estimate and admit only positive marginal gains without extra sampling, then distills the induced behavior into the base policy via a reverse-KL objective on its own empirical support. A co-evolutionary experience bank further synthesizes guidance from emerging failure modes and prunes obsolete entries as the policy evolves. Across embodied, web, and search-based QA tasks, EDGE improves over strong RL baselines by up to 12.5 points and remains effective without inference-time scaffolds or a proprietary reflector. The code is available at https://github.com/xvolcano02/EDGE.
Reliable cross-modal reasoning requires Multimodal Large Language Model (MLLMs) with interpretable and controllable computation. However, existing models rely on data-driven architectures that entangle multimodal processing, obscuring the functional processes underlying their outputs. Inspired by the human brain’s specialized and dynamically collaborative networks, we propose BrainMoE, a multimodal MoE model with six sensory and cognitive experts: Language, Logic, Social, World Knowledge, Vision, and Auditory. To support flexible sensory–cognitive coordination, we incorporates a layer-wise adaptive router that dynamically determines which experts are activated at each layer based on task-processing demands. Experiments on text, image, and audio reasoning benchmarks demonstrate that BrainMoE outperforms strong baselines by 1.59 and 2.2 points on text-only and multimodal benchmarks, while matching competitive external multimodal models. Routing and ablation analyses demonstrate clear expert specialization and causally important module-level contributions. Experiments on the cognitive benchmark further show that adaptive-routing BrainMoE aligns more closely with human behavioral patterns than baselines and static routing methods.