Navigating Intention Reasoning Trees to Guide Agentic Preference Optimization

Ziben Yue, Hongyu Yang, Weixin Li


Abstract
Understanding human intentions in videos requires deep cognitive reasoning, yet Multimodal Large Language Models (MLLMs) frequently suffer from factual hallucinations and counterfactual blindness. To address these limitations, we introduce ASTRO, a novel framework that navigates Intention Reansoning Tree to guide preference optimization. Our approach employs a collaborative multi-agent system integrated with Monte Carlo Tree Search (MCTS) to systematically explore complex reasoning spaces based on visual evidence. By navigating this tree, ASTRO dynamically extracts distinct cognitive trajectories—correct, error, and ambiguous paths—to construct high-quality preference pairs. Finally, we apply Direct Preference Optimization (DPO) using these contrastive paths, explicitly enhancing the model’s factual grounding and sensitivity to causal boundaries. Extensive experiments across complex video intention benchmarks demonstrate that ASTRO achieves superior performance while exhibiting remarkable data efficiency, significantly outperforming fully supervised baselines using only a fraction of the original training data.
Anthology ID:
2026.findings-emnlp.1740
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
37209–37228
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1740/
DOI:
10.18653/2026.findings-emnlp.1740
Bibkey:
Cite (ACL):
Ziben Yue, Hongyu Yang, and Weixin Li. 2026. Navigating Intention Reasoning Trees to Guide Agentic Preference Optimization. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 37209–37228, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Navigating Intention Reasoning Trees to Guide Agentic Preference Optimization (Yue et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1740.pdf