Multimodal sarcasm detection aims to identify sarcasm in image-text posts by capturing how visual content changes the interpretation of short informal text. Existing methods have advanced the task through multimodal fusion, caption-based cues, and fact/sentiment incongruity modeling, but their evidence use is often controlled inside latent feature channels or reduced discrepancy scores. This makes it difficult to decide which visual information should be selectively weighted for a particular post. We propose CAFE, a caption-anchored feature enrichment and evidence routing framework that decides how to use visual evidence for each sample under ambiguity. CAFE enriches each image with two generated descriptions, one for visible facts and one for the image-text relation, then estimates how useful each description is for the current post and fuses caption-calibrated contrast evidence with direct cross-modal evidence for prediction. Experiments show that CAFE achieves state-of-the-art performance on the primary MMSD2.0 benchmark with 87.81+/-0.33% F1 (mean over 5 seeds). Component ablations on the original MMSD provide an additional evaluation.