While reinforcement learning with verifiable rewards (RLVR) has advanced multimodal reasoning, coarse outcome rewards offer limited guidance for assigning credit to visual evidence. Analyzing same-prompt rollout groups, we observe that different outcomes exhibit distinct temporal patterns of measured visual dependency across reasoning segments. We therefore propose **VIGOR** (**Vi**sual **G**rounding **O**utcome-contrastive **R**einforcement), a policy optimization method that converts these group-wise contrasts into fine-grained supervision. VIGOR constructs outcome-conditioned prototypes from temporal visual fingerprints to reweight trajectories, uses segment-level contrasts to prioritize token gradients, and allocates the visual budget according to group difficulty. It requires no additional annotations or reward models and introduces no inference-time overhead. Across eight multimodal reasoning benchmarks, VIGOR achieves 57.5% average accuracy with Qwen2.5-VL-7B, improving over VPPO by 1.6 points. Controlled experiments further validate VIGOR on Qwen2.5-VL-3B and InternVL2.5-4B; on the latter, it outperforms VPPO on all eight tasks by 0.96 points on average. Our code is publicly available at https://github.com/beck83879-bit/VIGOR.