Online lifelong learning agents must decide not only how to act but also when to consult prior experience to continually improve on long-horizon tasks. Existing methods typically retrieve memories passively, such as at task initialization or after each step, and therefore miss knowledge gaps that arise during interaction. We propose ProactAgent, an experience-driven lifelong learning framework for proactive retrieval over a structured Experience Base. ProactAgent continually improves through ExpOnEvo, which jointly updates policies and refines memory, organizing past interactions into factual, episodic, and skill repositories. It further introduces ProactRL, which treats retrieval as an explicit policy action and learns when and what to retrieve. By comparing paired continuations from identical interaction prefixes with and without retrieval, ProactRL provides step-level process rewards that encourage retrieval only when it improves task outcomes or efficiency. Experiments on SciWorld, AlfWorld, and StuLife show that ProactAgent consistently outperforms all baselines, achieving up to 32% relative improvement in success rate and over 33% reduction in interaction rounds.

As AI research increasingly moves beyond systems optimized for static tasks, a central challenge is building agents that learn continuously through experience-driven lifelong learning (ELL). In this paper, we introduce StuLife, a synthetic benchmark designed to evaluate selected operational capabilities relevant to ELL. Particularly, StuLife simulates a student’s college journey, from enrollment to academic and personal development, across three core phases and ten detailed sub-scenarios. StuLife is designed around four key design principles: From Isolated Tasks to Stateful Interaction, From Passive to Proactive, From Context to Memory, and From Imitation to Learning. In this dynamic environment, agents must acquire and distill practical skills and maintain persistent memory to make decisions based on evolving state variables (e.g., resource availability and time), and are also expected to initiate actions without external prompting. To this end, StuLife provides an evaluation protocol with dedicated metrics (e.g., StuGPA) for these capabilities. Our evaluation reveals that even the highest-StuGPA model, GPT-5, scores only 17.9/100, highlighting large gaps in long-term retention and proactive task initiation. Beyond evaluating LLMs on StuLife, we also explore the role of context engineering in improving agent performance on lifelong learning tasks.