Large reasoning models improve difficult problem solving by spending many tokens on internal reasoning. This paradigm is natural for single-turn tasks, but poorly matched to language agents: an agent may take tens or hundreds of environment steps, making deep thinking before every action costly and sometimes harmful. We study agentic deep thinking: when should an agent invoke long-form private deliberation, and how should it be exposed inside a ReAct-style loop? We propose a selective deep-thinking framework that separates the short externalized Thought used to choose the next action from a longer private memo used to reorganize future behavior. Without model post-training, we instantiate this framework with Periodic Thinking, Sel.-Gate, and Sel.-Action. Across three benchmarks and six model families, selective deep thinking improves agents, but the best interface depends on backbone capability: strong proprietary models benefit most from deliberation as a callable action, while smaller open-source models are more robust with a separate gate. We then use GRPO to teach a less capable backbone when to deliberate, turning the weakest zero-shot interface into the strongest learned policy and improving transfer to unseen agentic tasks. These results show that selective deep thinking can improve both the effectiveness and efficiency of long-horizon language agents.