Long-horizon agents with structured actions execute tasks through sequences of dependent decisions, so a local decision mistake can change later observations and cause the task to fail. Stronger expert models offer a natural source of supervision, yet direct imitation or behavior cloning treats many differences between expert and base actions as equally useful targets. In this paper, we propose counterfactual selective imitation (CSI), a method that uses same-state counterfactual comparison to identify positive return differences. CSI assigns expert targets only when counterfactual rollouts show a high positive return difference, and routine states receive base-policy targets. The learned update is restricted to structured action spans, leaves the base model frozen, and does not require expert queries or replay at deployment. Experiments on AppWorld, ToolSandbox, and WebShop show that CSI raises average task success from 33.3% to 49.7% over the Qwen3-8B base and reduces average invalid/tool error from 9.3% to 4.9%. Compared with the strongest same-deployment adaptation baseline, CSI improves average task success by 1.3 points with only 5% additional deployed-inference latency.