@inproceedings{tian-etal-2026-egotools,
title = "{E}go{T}ools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos",
author = "Tian, Shulin and
Kim, Junsu and
Liu, Shuai and
Li, Hao and
Shen, Yujiao and
Li, Sihan and
Yang, Zhe and
Kim, Yeongon and
Yao, Runmao and
Dong, Yuhao and
Hong, Fangzhou and
Furnari, Antonino and
Yang, Jingkang and
Zhu, Hongyuan and
Liu, Ziwei",
editor = "Lee, Hung-Yi and
Dev, Sunipa and
Li, Jessy and
Elliott, Desmond",
booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.emnlp-main.961/",
pages = "22169--22190",
ISBN = "979-8-89176-459-0",
abstract = "Real-world embodied tasks require agents to act under physical constraints while tracking evolving object and task states. Because tools mediate many everyday and professional activities, understanding these tasks requires reasoning about affordances, hand{--}tool{--}object geometry, procedural progress, and causal effects. Tool-centric embodied reasoning therefore calls for real-world egocentric training data and diagnostic benchmarks that connect these abilities. We introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding. It combines EgoTools-Data, approximately 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information, with EgoTools-Bench, a diagnostic benchmark of 1,000 QA pairs across four tracks spanning perception, geometry, procedure, and causal reasoning. EgoTools-Bench reveals a substantial gap between current models and human experts. Under strict source-video separation, supervised fine-tuning of Qwen3-VL-8B-Instruct on instruction data derived from EgoTools-Data improves accuracy from 50.0{\%} to 60.9{\%} on the full 1,000-question benchmark. These results establish EgoTools as a unified training and diagnostic evaluation resource for real-world egocentric tool-use understanding."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tian-etal-2026-egotools">
<titleInfo>
<title>EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos</title>
</titleInfo>
<name type="personal">
<namePart type="given">Shulin</namePart>
<namePart type="family">Tian</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Junsu</namePart>
<namePart type="family">Kim</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Shuai</namePart>
<namePart type="family">Liu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hao</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yujiao</namePart>
<namePart type="family">Shen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sihan</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Zhe</namePart>
<namePart type="family">Yang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yeongon</namePart>
<namePart type="family">Kim</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Runmao</namePart>
<namePart type="family">Yao</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yuhao</namePart>
<namePart type="family">Dong</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Fangzhou</namePart>
<namePart type="family">Hong</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Antonino</namePart>
<namePart type="family">Furnari</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jingkang</namePart>
<namePart type="family">Yang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hongyuan</namePart>
<namePart type="family">Zhu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ziwei</namePart>
<namePart type="family">Liu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
</titleInfo>
<name type="personal">
<namePart type="given">Hung-Yi</namePart>
<namePart type="family">Lee</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sunipa</namePart>
<namePart type="family">Dev</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jessy</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Desmond</namePart>
<namePart type="family">Elliott</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-459-0</identifier>
</relatedItem>
<abstract>Real-world embodied tasks require agents to act under physical constraints while tracking evolving object and task states. Because tools mediate many everyday and professional activities, understanding these tasks requires reasoning about affordances, hand–tool–object geometry, procedural progress, and causal effects. Tool-centric embodied reasoning therefore calls for real-world egocentric training data and diagnostic benchmarks that connect these abilities. We introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding. It combines EgoTools-Data, approximately 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information, with EgoTools-Bench, a diagnostic benchmark of 1,000 QA pairs across four tracks spanning perception, geometry, procedure, and causal reasoning. EgoTools-Bench reveals a substantial gap between current models and human experts. Under strict source-video separation, supervised fine-tuning of Qwen3-VL-8B-Instruct on instruction data derived from EgoTools-Data improves accuracy from 50.0% to 60.9% on the full 1,000-question benchmark. These results establish EgoTools as a unified training and diagnostic evaluation resource for real-world egocentric tool-use understanding.</abstract>
<identifier type="citekey">tian-etal-2026-egotools</identifier>
<location>
<url>https://aclanthology.org/2026.emnlp-main.961/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>22169</start>
<end>22190</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos
%A Tian, Shulin
%A Kim, Junsu
%A Liu, Shuai
%A Li, Hao
%A Shen, Yujiao
%A Li, Sihan
%A Yang, Zhe
%A Kim, Yeongon
%A Yao, Runmao
%A Dong, Yuhao
%A Hong, Fangzhou
%A Furnari, Antonino
%A Yang, Jingkang
%A Zhu, Hongyuan
%A Liu, Ziwei
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F tian-etal-2026-egotools
%X Real-world embodied tasks require agents to act under physical constraints while tracking evolving object and task states. Because tools mediate many everyday and professional activities, understanding these tasks requires reasoning about affordances, hand–tool–object geometry, procedural progress, and causal effects. Tool-centric embodied reasoning therefore calls for real-world egocentric training data and diagnostic benchmarks that connect these abilities. We introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding. It combines EgoTools-Data, approximately 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information, with EgoTools-Bench, a diagnostic benchmark of 1,000 QA pairs across four tracks spanning perception, geometry, procedure, and causal reasoning. EgoTools-Bench reveals a substantial gap between current models and human experts. Under strict source-video separation, supervised fine-tuning of Qwen3-VL-8B-Instruct on instruction data derived from EgoTools-Data improves accuracy from 50.0% to 60.9% on the full 1,000-question benchmark. These results establish EgoTools as a unified training and diagnostic evaluation resource for real-world egocentric tool-use understanding.
%U https://aclanthology.org/2026.emnlp-main.961/
%P 22169-22190
Markdown (Informal)
[EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos](https://aclanthology.org/2026.emnlp-main.961/) (Tian et al., EMNLP 2026)
ACL
- Shulin Tian, Junsu Kim, Shuai Liu, Hao Li, Yujiao Shen, Sihan Li, Zhe Yang, Yeongon Kim, Runmao Yao, Yuhao Dong, Fangzhou Hong, Antonino Furnari, Jingkang Yang, Hongyuan Zhu, and Ziwei Liu. 2026. EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 22169–22190, Budapest, Hungary. Association for Computational Linguistics.