Agentic RL can benefit substantially from reusable experience, but existing skill-based methods mostly focus on trajectory-level guidance and provide limited support for the principled evaluation and maintenance of an evolving skill memory. We propose D2Skill, a dynamic dual-granularity skill bank for agentic RL that organizes reusable experience into task skills for high-level guidance and step skills for fine-grained decision support and error correction. D2Skill jointly trains the policy and skill bank through paired baseline and skill-injected rollouts under the same policy, using their performance gap to derive hindsight utility signals for both skill updating and policy optimization. Constructed from training-time trajectories, the skill bank is continuously expanded through reflection and maintained with utility-aware retrieval and pruning. Across ALFWorld, WebShop, and Search-Augmented QA, D2Skill consistently outperforms skill-free and skill-based baselines, remains effective with self-feedback, and transfers to unseen ALFWorld environments without test-time adaptation. Project Page: https://github.com/TU2021/D2Skill-AgenticRL