Streaming Atomic Action Segmentation in Egocentric Videos via Iterative LVLM-to-Streaming Distillation

Hyolim Kang, Jungsoo Lee, Janghoon Cho, Juntae Lee, Seon Joo Kim, Sungha Choi


Abstract
Large Vision-Language Models (LVLMs) are promising scalable annotators for video understanding, but they are not directly suited for dense temporal annotation. We study this challenge in egocentric streaming understanding, where the goal is to segment continuous first-person video into atomic actions: short, concrete physical interactions with well-localized temporal boundaries.

We propose CoTT (Chains of Temporal Transitions), an annotation-free framework that structures LVLM reasoning around temporal transitions and combines it with iterative streaming distillation. CoTT uses structured LVLM reasoning to generate dense atomic action pseudo-labels from unlabeled egocentric videos, distills these labels into a lightweight online model, and bootstraps the model’s predictions to improve temporal consistency.

On large-scale egocentric benchmarks, EPIC-Kitchens-100 and Ego-Exo4D, CoTT achieves competitive performance against supervised online localization methods without using ground truth temporal annotations for training. It also transfers across egocentric datasets and, on THUMOS14, outperforms prior best training-free zero-shot Temporal Action Localization methods, even without any dataset-specific training or adaptation.
Anthology ID:
2026.emnlp-main.1163
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
26613–26631
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1163/
DOI:
Bibkey:
Cite (ACL):
Hyolim Kang, Jungsoo Lee, Janghoon Cho, Juntae Lee, Seon Joo Kim, and Sungha Choi. 2026. Streaming Atomic Action Segmentation in Egocentric Videos via Iterative LVLM-to-Streaming Distillation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 26613–26631, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Streaming Atomic Action Segmentation in Egocentric Videos via Iterative LVLM-to-Streaming Distillation (Kang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1163.pdf