Hateful meme detection remains a formidable challenge for vision-language models, as existing benchmarks are structurally observational—confounding rhetorical hate mechanisms with target community features and preventing causal evaluation of model vulnerabilities. To address this, we introduce **FBHM**, a systematically curated benchmark of ***F****unctionality* ***B****ased* ***H****ateful* ***M****emes* constructed along two orthogonal axes: 25 distinct rhetorical functionalities and 10 target communities (5,000 memes total). Benchmarking state-of-the-art VLMs reveals a severe generalization gap: models highly accurate on standard datasets catastrophically drop to near-random performance on **FBHM**, proving they exploit dataset-specific heuristics rather than robust multimodal reasoning. To efficiently close this gap, we propose **LSV** (***l****earnable* ***s****t*eering ***v****ectors*), an ultra-low data regime strategy that applies a causal intervention objective on as few as 500 steering samples (50 unique base memes), boosting **FBHM** performance by ~30 Macro-F1 points while outperforming in-context learning and PEFT without degrading source-domain performance.