Wentao Gao

Author directory

2026

Multimodal agents are increasingly evaluated on their ability to search the web, invoke tools, and solve complex tasks autonomously. Yet no existing benchmark systematically evaluates the capability of autonomously searching, parsing, and extracting precise answers from multimodal raw media files as a unified, end-to-end agentic task. We introduce RDQA - Raw Data Question Answering, a benchmark of 1,350 questions across three source modalities (document, video, audio), and 20 real-world task scenarios, where every answer is designed to require source access rather than model knowledge alone. RDQA provides capability-stratified evaluation through 9 capability families organized by modality and cognitive complexity, and introduces the Parametric Leakage Rate (PLR), a process-aware metric that flags correct answers produced without genuine evidence retrieval. We evaluate 12 agent systems spanning 6 backbone models and 2 scaffoldings. Our results show that the the best generic agent reaches only 24.0% adjusted accuracy, while a process-aware baseline (RDQA-Skill) reaches 41.1%. PLR reaches as high as 96.0%, indicating that many ostensibly correct answers are not grounded in retrieved evidence. We release the full dataset, evaluation framework, and a baseline to support future research.
Medical Vision-Language Models (VLMs) are vulnerable to misleading clinical prompts and may follow incorrect user assertions instead of image evidence, a behavior commonly referred to as sycophancy. Existing mitigation methods mainly rely on prompt-level constraints to encourage neutral responses. However, through fine-grained integrated gradient analysis, we observe a phenomenon, although prompting strategies can correct the final prediction, the internal multimodal representations remain biased toward text tokens, while visual tokens continue to receive limited attention in late fusion layers. To address this issue, we propose HD-AS (Head-Driven Activation Steering), a training-free inference-time intervention framework. HD-AS identifies sycophancy-related attention heads through activation drift analysis under different prompting conditions. During inference, a lightweight isotropic damping operation is applied through a forward pre-hook on the target projection layers, reducing the dominance of text features in the hidden space and restoring the contribution of visual representations. Experiments on two public medical Visual Question Answering (VQA) datasets show that HD-AS consistently reduces the flip rate while maintaining diagnostic accuracy.