Complex decision-making often requires agents to progress through intermediate tasks rather than solve the final target directly. Existing LLM self-refinement methods typically iterate on a fixed target, while curriculum-learning methods often rely on hand-designed schedules, training-time optimization, or domain-specific difficulty metrics. To smooth the learning curve through adaptive curriculum design, we introduce EvoCurr, a general inference-time framework that co-evolves curricula and executable policies. A Designer proposes verifiable intermediate tasks from the latest accepted progress and recent failures, while a Solver generates or trains policies for these tasks. A task-policy pair is accepted only when it passes hard feasibility checks and reaches a specified performance threshold; meanwhile, an accepted-floor constraint preserves the latest verified checkpoint and prevents failed harder attempts from overwriting mastered behavior. We evaluate EvoCurr across code-as-policy and closed-loop MARL settings, using StarCraft II super-late-game micromanagement as our hardest stress test. EvoCurr demonstrates effective micromanagement across three LLM backbones without LLM parameter training. Improvements in additional experiments on Flatland, POGEMA, Overcooked, and LiveCodeBench Hard demonstrate the generality of EvoCurr.