Haozhe Luo
Author directoryOther people with similar names: HaoZhe Luo
2026
SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos
Zheyu Huang | Zijing Shi | Haozhe Luo | Huadong Tang | Mingyu Liu | Meng Fang | Ling Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Zheyu Huang | Zijing Shi | Haozhe Luo | Huadong Tang | Mingyu Liu | Meng Fang | Ling Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent advances in Large Multimodal Models (LMMs) have greatly improved video understanding, yet their ability to reason about human-centered social situations remains limited. Existing benchmarks typically rely on videos with a single observed trajectory, making it difficult to determine whether models truly understand social dynamics or merely exploit recurring narrative patterns. We introduce SocialReasonBench, a video multiple-choice QA benchmark for evaluating socially grounded reasoning in scenarios derived from interactive narratives. Built from gameplay videos of Detroit: Become Human, the benchmark leverages branching storylines where player decisions lead to alternative social outcomes that can be checked against the game’s own script, flowchart, and recorded branches. We develop a multi-agent curation pipeline that localizes socially meaningful clips, grounds answer labels in game-state signals, and generates theory-guided questions with diagnostic distractors. SocialReasonBench covers seven reasoning dimensions, including intent recognition, emotional empathy, moral dilemma, counterfactual reasoning, and causal antecedent. Experiments on contemporary LMMs show that models perform reasonably well on basic social understanding but struggle with counterfactual and causal reasoning. Further ablation and diagnostic error analyses reveal that models often depend on incomplete modality cues and fall into reasoning traps such as visual shortcuts, highlighting a gap between observable event recognition and deeper reasoning over latent social states.
Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
Zirui Song | Yuan Huang | Junchang Liu | Haozhe Luo | Chenxi Wang | Lang Gao | Zixiang Xu | Bairui Zhang | Mingfei Han | Xiaojun Chang | Xiuying Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zirui Song | Yuan Huang | Junchang Liu | Haozhe Luo | Chenxi Wang | Lang Gao | Zixiang Xu | Bairui Zhang | Mingfei Han | Xiaojun Chang | Xiuying Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Social deduction games like Werewolf combine language, reasoning, and strategy, providing a testbed for studying language and social intelligence. However, most studies reduce the game to LLM-based self-play, yielding templated utterances and anecdotal cases that overlook the richness of social gameplay. Evaluation further relies on coarse metrics such as survival time or subjective scoring due to the lack of quality reference data. To address these gaps, we curate a high-quality, human-verified, multilingual, and multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants. Based on this dataset, we propose a novel strategy-alignment evaluation that leverages the winning faction’s strategies as ground truth in two stages: 1) Speech evaluation, formulated as multiple-choice-style tasks that assess whether the model can adopt appropriate stances across five dimensions of social ability. 2) Decision evaluation, which assesses the model’s voting choices and opponent-role inferences. This framework enables a fine-grained evaluation of models’ linguistic and reasoning capabilities, while capturing their ability to generate strategically coherent gameplay. Our experiments show that state-of-the-art LLMs and VLMs exhibit diverse performance, with roughly half of the models in both modalities remaining below 0.50 accuracy, revealing clear gaps in deception and counterfactual reasoning.