Hideo Saito

Author directory

2026

Recent advances in image and video generation have highlighted the need for intuitive, high-precision manipulation. However, traditional text-based prompts or 2D Visual Prompts (e.g., dragging, 2D trajectories) often suffer from unintuitive prompt engineering or depth ambiguity. To address these limitations, we introduce 3D Visual Prompting, a novel paradigm that significantly improves input intuitiveness for visual manipulation tasks. We develop a framework that integrates intuitive 3D constraints into 2D generative processes. Our approach projects 3D Visual Prompts onto 2D visual planes and leverages Multimodal Large Language Models (MLLMs) to verbalize intent, allowing plug-and-play use of existing 2D models. Furthermore, we incorporate In-Context Self-Refinement (ICSR) to summarize insights from self-correction into a meta-strategy. This enables high-precision 3D control in single-pass inference, eliminating the need for iterative computation. Experimental results demonstrate that our method significantly outperforms traditional 2D Visual Prompting for object translation and rotation. By utilizing pre-trained models without additional parameter tuning, our framework remains highly adaptable to future developments in generative models.