Yingying Yao

Author directory

2026

With the explosive growth of social media, multimodal memes (text and images) have become a key carrier of misinformation dissemination and emotional manipulation because they strongly depend on cultural background, satirical metaphor, and current events. Existing detection methods mostly rely on end-to-end deep learning models, but they face two fundamental limitations: first, static models are difficult to capture the dynamic semantic evolution of memes; second, fixed architectures cannot adapt to meme heterogeneity. This work proposes a Tool-Learning Framework for Multimodal Meme Detection (Tool-meme), a Large Multimodal Model (LMM)-driven tools-learning framework. The framework uses multimodal representations only as input features, dynamically orchestrates tool chains through retrieval-augmented generation and meta-cognitive planning, and uses reinforcement learning to maximize tool-use efficiency. On HarM, FHM, and MAMI, Tool-meme outperforms all evaluated direct-LMM and agentic baselines under frozen-executor setting. Additional transfer and deployment analyses show consistent gains across tested executor families while lowering tool calls and cost vs. exhaustive execution, with explicit evidence chains for inspection.