Recent advances in image and video generation have highlighted the need for intuitive, high-precision manipulation. However, traditional text-based prompts or 2D Visual Prompts (e.g., dragging, 2D trajectories) often suffer from unintuitive prompt engineering or depth ambiguity. To address these limitations, we introduce 3D Visual Prompting, a novel paradigm that significantly improves input intuitiveness for visual manipulation tasks. We develop a framework that integrates intuitive 3D constraints into 2D generative processes. Our approach projects 3D Visual Prompts onto 2D visual planes and leverages Multimodal Large Language Models (MLLMs) to verbalize intent, allowing plug-and-play use of existing 2D models. Furthermore, we incorporate In-Context Self-Refinement (ICSR) to summarize insights from self-correction into a meta-strategy. This enables high-precision 3D control in single-pass inference, eliminating the need for iterative computation. Experimental results demonstrate that our method significantly outperforms traditional 2D Visual Prompting for object translation and rotation. By utilizing pre-trained models without additional parameter tuning, our framework remains highly adaptable to future developments in generative models.

Recent action recognition based on vision–language pretraining and self-supervised video foundation models tends to induce spurious correlations and shortcut learning by relying on action-irrelevant cues, such as backgrounds and object co-occurrences. By contrast, object-detection-based approaches can suppress spurious correlations; however, the loss of input information can limit accuracy. To mitigate this trade-off, we combine these two approaches to learn complementary features that compensate for each other’s shortcomings. Specifically, we leverage the commonsense knowledge of large language models (LLMs) regarding human actions and realize a framework in which an LLM agent integrates the two approaches within an agentic learning paradigm to design motion features tailored to the target actions. The LLM agent uses an open vocabulary object detector to instruct the video foundation model with the target and nontarget objects in the video to make the model attend to objects in a video required for recognizing the target actions. The composition of the detected objects is optimized for the target actions through in-context reinforcement learning (ICRL) using the commonsense knowledge of the LLM. Experiments on multiple public action recognition datasets and an ablation study confirm the robustness of features learned using the proposed method and the effectiveness of ICRL.