Haomin Tan

Author directory

2026

Existing multimodal sentiment analysis (MSA) methods rely on representation learning and cross-modal fusion. However, many follow a correlation-driven paradigm that may encode lexical shortcuts, visual biases, and non-causal correlations into predictions. To address this, we propose a Consistency-Conflict Fusion with Sample-Adaptive Causal Evidence (CCF-SACE) framework, revisiting MSA from a representation-level causal modeling perspective. First, encoders extract textual and visual features, and an adaptor projects them into a shared space. The framework then performs sample-adaptive cross-modal causal evidence selection, where textual causal evidence is selected under visual context and visual causal evidence under textual context. The remaining contexts are modeled as non-causal proxies, allowing the model to separate emotion-supportive evidence from shortcut-related information. CCF-SACE further introduces a covariance-based consistency-conflict-aware fusion module that constructs an evidence-level mediator by modeling cross-modal consistency and discrepancy rather than directly fusing all selected cues. Since the residual proxy stream may still carry label-correlated shortcut information, we introduce non-causal proxy suppression and counterfactual sensitivity objectives to weaken its standalone predictability and enlarge the discrepancy between evidence-mediated and proxy-only predictions. Experiments on five benchmarks show consistent improvements over state-of-the-art methods, especially on fine-grained affective recognition.