Chuhao Zhou
Author directory2026
Routing-Aware Head Amplification for Multimodal Long-Chain-of-Thought Reasoning
Chuhao Zhou | Zonghan Wu | Haolang Lu | Kun Wang | Weiran Yao | Linsey Pang | Prakhar Mehrotra | Qingsong Wen | Yilei Shao
Findings of the Association for Computational Linguistics: EMNLP 2026
Chuhao Zhou | Zonghan Wu | Haolang Lu | Kun Wang | Weiran Yao | Linsey Pang | Prakhar Mehrotra | Qingsong Wen | Yilei Shao
Findings of the Association for Computational Linguistics: EMNLP 2026
Multimodal long-chain-of-thought (CoT) generation requires models to dynamically route information between visual evidence and an expanding reasoning history. Existing hallucination mitigation methods often strengthen visual grounding or intervene in decoding, but typically treat grounding as a relatively static property of generation. We propose a training-free dynamic attention intervention framework that exploits the head-level sparsity, layer-level stratification, and trajectory-level dynamics of multimodal long-CoT decoding. Our method selects routing-sensitive heads using perturbation-only calibration, assigns visual-evidence and reasoning-history interventions to different layer regions, and continuously rescales selected head outputs according to their instantaneous token-group attention ratios. Across multiple MLLM architectures and multimodal hallucination benchmarks, our method consistently improves over vanilla decoding and visual contrastive decoding, achieving an average performance gain of 4.6%.