Think-with-images, where multimodal LLMs (MLLMs) actively manipulate visual inputs as part of their reasoning, has become a focal research direction for agentic multimodal systems, with a surge of prompting, supervised, and reinforcement learning (RL) methods teaching models to invoke vision tools such as cropping, zooming, and editing. Yet most established multimodal benchmarks use images whose critical content is already easy to perceive in a single pass, and many have saturated on frontier models. We introduce VisualToolBench (VTB), a benchmark deliberately curated so that critical visual content is dense, occluded, dispersed, rotated, or low-contrast, making active image manipulation a prerequisite for success. VTB contains 1,204 expert-authored, open-ended tasks (603 single-turn, 601 multi-turn) across five domains. Each task is paired with rubrics tagged by five competency categories (Visual Understanding, Truthfulness, Instruction Following, Reasoning, and Writing Style) that enable per-axis disentanglement. Across 26 evaluated MLLMs, even the strongest model reaches only 29% accuracy, with Visual Understanding the consistent bottleneck and visual misinterpretation accounting for 32% to 39% of failures. Top models, moreover, converge to comparable accuracy through markedly different tool-use strategies, which we analyze in detail.