Bo Peng
Author directoryOther people with similar names: Bo Peng, Bo Peng, Bo Peng, Peng Bo
Unverified author pages with similar names: Bo Peng
2026
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
Zhiheng Wang | Bo Peng | Lai Wei | Chaochao Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhiheng Wang | Bo Peng | Lai Wei | Chaochao Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
The “thinking-with-images” paradigm equips multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations often achieve only marginal or negative gains over direct inference at substantially higher token cost. They may also repeatedly crop irrelevant regions and fail on questions that direct inference answers correctly. We ask whether the returned visual evidence causally affects the answer. To answer this question, we formulate visual tool-use as a causal graph that separates observation-mediated paths from action-induced shortcuts. We then audit it through interventions at the three levels: policy (comparing tool-use with direct inference), trajectory (corrupting all observations during rollout), and step (counterfactually replacing one individual observation under a fixed prefix). Our step-level estimand, Visual Evidence Gain, isolates the contribution of each returned observation. Across six representative models and five fine-grained perception benchmarks, we uncover policy miscalibration with two failure modes. In Calling Without Looking, returned observations have no causal effect on the answer. In Looking Without Planning, observations are informative but the call schedule is incoherent. A trajectory-level diagnostic decomposes the policy-level accuracy gain into per-group contributions and shows that the gain is concentrated in a Calibrated minority. We term this discrepancy the illusion of visual tool-use: despite aggregate accuracy gains, visual tool-use is not causally effective across a broad range of rollouts. The code is available at https://github.com/OpenCausaLab/CauAudit.
CauSight: Learning to Supersense for Visual Causal Discovery
Yize Zhang | Meiqi Chen | Sirui Chen | Bo Peng | Yanxi Zhang | Tianyu Li | Chaochao Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yize Zhang | Meiqi Chen | Sirui Chen | Bo Peng | Yanxi Zhang | Tianyu Li | Chaochao Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Causal thinking enables humans to understand not just what is seen, but why it happens. To bring this ability to modern AI systems, we introduce visual causal discovery, the task of inferring visually verifiable direct causal dependencies among entities in a single static image. We construct the Visual Causal Graph dataset (VCG-32K), a large-scale collection of over 32,000 images annotated with entity-level causal graphs, and develop Causight, a causally aware vision-language model for this task. Our training recipe combines data curation from VCG-32K, Tree-of-Causal-Thought (ToCT) reasoning synthesis, and reinforcement learning with a graph-structured causal reward. Experiments show that Causight outperforms GPT-4.1, achieving over a threefold performance boost (21% absolute gain).
2025
IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data
Bo Peng | Zhiheng Wang | Heyang Gong | Chaochao Lu
Findings of the Association for Computational Linguistics: EMNLP 2025
Bo Peng | Zhiheng Wang | Heyang Gong | Chaochao Lu
Findings of the Association for Computational Linguistics: EMNLP 2025
In modern dialogue systems, the ability to implicitly infer user backgrounds from conversations and leverage this information for personalized assistance is crucial. However, the scarcity of high-quality data remains a fundamental challenge to evaluating and improving this capability. Traditional dataset construction methods are labor-intensive, resource-demanding, and raise privacy concerns. To address these issues, we propose a novel approach for automatic synthetic data generation and introduce the Implicit Personalized Dialogue (IP-Dialog) benchmark along with a training dataset, covering 10 tasks and 12 user attribute types. Additionally, we develop a systematic evaluation framework with four metrics to assess both attribute awareness and reasoning capabilities. We further propose five causal graphs to elucidate models’ reasoning pathways during implicit personalization. Extensive experiments yield insightful observations and prove the reliability of our dataset.
2024
CELLO: Causal Evaluation of Large Vision-Language Models
Meiqi Chen | Bo Peng | Yan Zhang | Chaochao Lu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Meiqi Chen | Bo Peng | Yan Zhang | Chaochao Lu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Causal reasoning is fundamental to human intelligence and crucial for effective decision-making in real-world environments. Despite recent advancements in large vision-language models (LVLMs), their ability to comprehend causality remains unclear. Previous work typically focuses on commonsense causality between events and/or actions, which is insufficient for applications like embodied agents and lacks the explicitly defined causal graphs required for formal causal reasoning. To overcome these limitations, we introduce a fine-grained and unified definition of causality involving interactions between humans and/or objects. Building on the definition, we construct a novel dataset, CELLO, consisting of 14,094 causal questions across all four levels of causality: discovery, association, intervention, and counterfactual. This dataset surpasses traditional commonsense causality by including explicit causal graphs that detail the interactions between humans and objects. Extensive experiments on CELLO reveal that current LVLMs still struggle with causal reasoning tasks, but they can benefit significantly from our proposed CELLO-CoT, a causally inspired chain-of-thought prompting strategy. Both quantitative and qualitative analyses from this study provide valuable insights for future research. Our project page is at https://github.com/OpenCausaLab/CELLO.