Zihao Zhou
Author directoryOther people with similar names: Zihao Zhou
Unverified author pages with similar names: Zihao Zhou
2026
Transferring Math Reasoning Abilities from LLMs to MLLMs via Causal-Aware Model Merging
Yijie Hu | Guanyu Yang | Zihao Zhou | Bin Dong | Xiaowei Huang | Kaizhu Huang | Qiufeng Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Yijie Hu | Guanyu Yang | Zihao Zhou | Bin Dong | Xiaowei Huang | Kaizhu Huang | Qiufeng Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
To build a math MLLM, current research efforts predominantly rely on elaborate training schemes or high-quality multi-modal math reasoning data. On the other hand, model merging has attracted much attention recently due to its low-cost and tune-free property, which has shown great potential for constructing math MLLM. However, most recent attempts either assume all parameters are task-relevant or adopt a less interpretable parameter selection strategy, thus merging parameters on a coarse granularity, leading to limited or even degrading performance. In this paper, we propose a causal-analysis–guided parameter merging framework, which identifies the math reasoning-related parameters in a finer granularity and interpretable manner, then merges the math LLMs with MLLM. Specifically, we first construct targeted arithmetic and logical tasks, generate reference and counterfactual samples, and then apply intervention-based causal attribution to localize parameters that are causally responsible for math reasoning. Once we identify the crucial reasoning parameters in the math LLM that exhibit higher causal effects, we can more precisely align and merge only the corresponding parameters from the math LLM into the base MLLM. The procedure is tuning-free and enhances both math reasoning and general reasoning abilities. Experimental results of three model families on four math reasoning benchmarks and two general reasoning benchmarks validate the effectiveness of the proposed approach.
VisMind: Stateful Synthetic Dialogue for Visual Mid-Training
Xinlong Fu | Zihao Zhou | Qiufeng Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Xinlong Fu | Zihao Zhou | Qiufeng Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
The mid-training phase has emerged as a critical bridge for enhancing the reasoning capabilities of Vision-Language Models, yet its effectiveness is constrained by the scarcity of high-quality visual reasoning data. While data synthesis offers a promising alternative, conventional stateless synthesis paradigms produce shallow trajectories that quickly succumb to utility plateaus. To overcome this, we propose VisMind, a novel multi-agent collaborative framework that generates deep-reasoning Stateful Synthetic Dialogue (SSD) from visual inputs. By orchestrating iterative problem-solving, where peer student agents engage in bidirectional critique under the ground-truth-anchored guidance of a teacher, VisMind transforms static, fragmented responses into rich, multi-turn logical derivations. Utilizing this interactive paradigm, we construct a dense mid-training corpus of approximately 1.14B tokens. Extensive experiments across the Qwen2-VL and InternVL3 model families demonstrate that applying this corpus via Continued Pretraining consistently outperforms a representative stateless synthesis approach and elevates standard baselines by up to a 2.24% absolute average gain across comprehensive mathematical and logical benchmarks. Furthermore, in-depth analysis reveals that the SSD paradigm exhibits continuous linear scaling without early saturation, fosters intrinsic reasoning generalization, and establishes a robust cognitive foundation that enhances the efficacy of subsequent Reinforcement Learning with Verifiable Rewards.