Qinwu Xu
Author directory2026
Human-Perception-Aware but Not Yet Physically Grounded: Strengths and Failure Modes of Frontier Multimodal LLMs and the GroundVQA Benchmark
Qinwu Xu | Xin Liu | Yifan Jiang
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Qinwu Xu | Xin Liu | Yifan Jiang
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Frontier multimodal large language models (MLLMs) have achieved strong performance in visual question answering (VQA), OCR-intensive image understanding, and multimodal reasoning. However, they can still produce linguistically plausible answers that are not supported by visual evidence, particularly under ambiguous OCR, weak visual evidence, and challenging spatial or physical reasoning. We introduce GroundVQA, an adversarial VQA benchmark designed to evaluate visual grounding, hallucination resistance, OCR robustness, uncertainty handling, and spatial and physical reasoning. GroundVQA contains 3,877 manually reviewed question–answer pairs constructed from approximately 2,400 public-source images and around 100 self-collected real-world images. We evaluate representative frontier MLLMs on a stratified subset of GroundVQA and complement the quantitative evaluation with qualitative analyses of recurring model strengths and failure modes. Our findings highlight a persistent reliance on semantic priors when visual evidence is ambiguous or insufficient, motivating evaluation frameworks that emphasize grounded reasoning, calibrated uncertainty, and adversarial premise rejection beyond conventional VQA accuracy.