Fangzhou Hong
Author directory2026
EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos
Shulin Tian | Junsu Kim | Shuai Liu | Hao Li | Yujiao Shen | Sihan Li | Zhe Yang | Yeongon Kim | Runmao Yao | Yuhao Dong | Fangzhou Hong | Antonino Furnari | Jingkang Yang | Hongyuan Zhu | Ziwei Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Shulin Tian | Junsu Kim | Shuai Liu | Hao Li | Yujiao Shen | Sihan Li | Zhe Yang | Yeongon Kim | Runmao Yao | Yuhao Dong | Fangzhou Hong | Antonino Furnari | Jingkang Yang | Hongyuan Zhu | Ziwei Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Real-world embodied tasks require agents to act under physical constraints while tracking evolving object and task states. Because tools mediate many everyday and professional activities, understanding these tasks requires reasoning about affordances, hand–tool–object geometry, procedural progress, and causal effects. Tool-centric embodied reasoning therefore calls for real-world egocentric training data and diagnostic benchmarks that connect these abilities. We introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding. It combines EgoTools-Data, approximately 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information, with EgoTools-Bench, a diagnostic benchmark of 1,000 QA pairs across four tracks spanning perception, geometry, procedure, and causal reasoning. EgoTools-Bench reveals a substantial gap between current models and human experts. Under strict source-video separation, supervised fine-tuning of Qwen3-VL-8B-Instruct on instruction data derived from EgoTools-Data improves accuracy from 50.0% to 60.9% on the full 1,000-question benchmark. These results establish EgoTools as a unified training and diagnostic evaluation resource for real-world egocentric tool-use understanding.