The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences

Virmarie Maquiling, Enkelejda Kasneci


Abstract
Human gaze is the primary high-bandwidth interface through which we parse the semantics of our environment. While often modeled as a sequence, the degree to which gaze exhibits transferable local structure across domains remains underexplored. Using motion tokenization as a diagnostic lens across four diverse datasets, we investigate the transferability of local sequence structure. We demonstrate that while gaze can share a reusable motion-token vocabulary, identical motion tokens are organized into different local transition patterns depending on the dataset and task context. Our results show the limitations of naive data-pooling under low-semantic motion-token representations and provide a framework for evaluating behavioral sequences through information-theoretic diagnostics.
Anthology ID:
2026.findings-emnlp.405
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
8564–8581
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.405/
DOI:
Bibkey:
Cite (ACL):
Virmarie Maquiling and Enkelejda Kasneci. 2026. The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 8564–8581, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences (Maquiling & Kasneci, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.405.pdf