Practical deployment of medical vision-language models requires a single model that interprets heterogeneous clinical visual inputs: 2D images, 3D volumes, and medical videos. Yet how to train such unified models remains underexplored, as these modalities differ in spatial-temporal structure, token length, and data availability. We present OmniV-Med, a scalable training recipe built on four empirical findings. First, broad modality coverage requires balancing modality diversity with task-level supervision, motivating OmniV-Med-Instruct, a 252K corpus spanning 14 medical modalities and 11 clinical tasks. Second, we identify a modality-ordering effect: introducing 3D and video too early destabilizes optimization and weakens 2D grounding, motivating a progressive curriculum that first builds 2D alignment before extending to spatial-temporal inputs. Third, we observe an emergent spatiotemporal transfer effect: once strong 2D patch representations are in place, lightweight rotary position adaptation extrapolates them to 3D slices and video frames, letting a single 2D ViT replace modality-specific encoder stacks. Fourth, the imbalance in sequence length across modalities and the redundancy of long volumes and videos hinder scaling; our Redundancy-Guided Token Pruning (RGTP) exploits inter-slice and inter-frame redundancy to cut visual tokens by 60% while preserving performance. Across nine 2D, 3D, and video medical benchmarks, OmniV-Med achieves robust performance. Extensive component-wise ablations systematically refine the design space, yielding a simple yet efficient and scalable recipe for training high-performance unified Med-VLMs. Code is released at https://github.com/ZJUI-AI4H/Hulu-Med.

Medical Visual Question Answering (Med-VQA), which offers language responses to image-based medical inquiries, represents a challenging task and significant advancement in healthcare. It assists medical experts to swiftly interpret medical images, thereby enabling faster and more accurate diagnoses. However, the model interpretability and transparency of existing Med-VQA solutions are often limited, posing challenges in understanding their decision-making processes. To address this issue, we devise a semi-automated annotation process to streamline data preparation and build new benchmark Med-VQA datasets R-RAD, R-SLAKE and R-Path. These datasets provide intermediate medical decision-making rationales generated by multimodal large language models and human annotations for question-answering pairs in existing Med-VQA datasets, i.e., VQA-RAD, SLAKE and PathVQA. Moreover, we design a novel framework, MedThink, which finetunes lightweight pretrained generative models by incorporating medical decision-making rationales. MedThink includes three distinct strategies to generate decision outcomes and corresponding rationales, clearly showcasing the medical decision-making process during reasoning. Our comprehensive experiments show that our method achieves an accuracy of 83.5% on R-RAD, 86.3% on R-SLAKE and 87.2% on R-Path. These results significantly exceed those of existing state-of-the-art models with comparable parameters. Datasets and code are available at https://github.com/Tang-xiaoxiao/Medthink.

Multimodal large language models (MLLMs) still struggle with complex reasoning tasks in Visual Question Answering (VQA). While current methods have advanced by incorporating visual prompts, our study uncovers critical limitations: these approaches indiscriminately annotate all detected objects for every visual question, generating excessive visual markers that degrade task performance. This issue stems primarily from a lack of focus on key visual elements, raising two important questions: Are all objects equally important, and do all questions require visual prompts? Motivated by Dual Process Theory, which distinguishes between instinctive and deliberate cognitive modes in human reasoning, we propose FOCUS, a plug-and-play approach that dynamically adapts to the complexity of questions, combining fast intuitive judgments with deliberate analytical reasoning to enhance the vision-language reasoning capability of the MLLM. For straightforward questions, FOCUS supports efficient zero-shot reasoning. For more complex tasks, it employs the conceptualizing before observation strategy to highlight critical elements. Extensive experiments on four benchmarks—ScienceQA, TextQA, VizWiz, and MME—demonstrate that FOCUS consistently improves the performance of both open-source and black-box MLLMs, achieving significant gains across all datasets. Ablation studies further validate the importance of combining diverse cognitive strategies with refined visual information for superior performance. Code will be released.

Recent advancements in large language models (LLMs) have highlighted their potential across a variety of tasks, but their performance still heavily relies on the design of effective prompts. Existing methods for automatic prompt optimization face two challenges: lack of diversity, limiting the exploration of valuable and innovative directions and semantic drift, where optimizations for one task can degrade performance in others. To address these issues, we propose Residual Optimization Tree (RiOT), a novel framework for automatic prompt optimization. RiOT iteratively refines prompts through text gradients, generating multiple semantically diverse candidates at each step, and selects the best prompt using perplexity. Additionally, RiOT incorporates the text residual connection to mitigate semantic drift by selectively retaining beneficial content across optimization iterations. A tree structure efficiently manages the optimization process, ensuring scalability and flexibility. Extensive experiments across five benchmarks — covering commonsense, mathematical, logical, temporal, and semantic reasoning — demonstrate that RiOT outperforms both previous prompt optimization methods and manual prompting. Code will be released.