Annotating and Recognizing the Palm Up Open Hand Gesture in Dutch

Fenna Berger, Alan Cienki, Lucia Donatelli


Abstract
Recurrent gestures are conventionalized gestures with a stable relationship between form and meaning. One such gesture is the palm up open hand (PUOH) gesture, typically produced by extending the fingers and rotating the lower arm so that the palm faces upward. The gesture is attested across many languages and cultures and is often used to present a point, yet little is known about the forms and functions it takes in Dutch. Automatic gesture recognition could reduce the manual labor that quantitative gesture studies require, but recognizing this gesture is difficult, partly because of the large amount of variation present in its form. This paper compares two gesture recognition approaches on a newly collected corpus of naturalistic Dutch dyadic conversation: a static model that takes single images as input and a dynamic model that takes video clips as input. Both models are trained and tested with varying amounts of gesture-form variation included in the data. The static model is found to be less affected by this variation and to yield overall better results than the dynamic model. Our results illustrate some of the central challenges that gesture recognition systems face in recognizing the PUOH gesture and offer insights that may improve performance in future studies of multimodal, face-to-face behavior.
Anthology ID:
2026.mint-1.16
Volume:
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Esam Ghaleb, Ece Takmaz, Diego Frassinelli, Bulat Khaertdinov, Raquel Fernández, Zerrin Yumak, Asli Ozyurek
Venues:
MINT | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
150–159
Language:
URL:
https://aclanthology.org/2026.mint-1.16/
DOI:
Bibkey:
Cite (ACL):
Fenna Berger, Alan Cienki, and Lucia Donatelli. 2026. Annotating and Recognizing the Palm Up Open Hand Gesture in Dutch. In Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue, pages 150–159, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Annotating and Recognizing the Palm Up Open Hand Gesture in Dutch (Berger et al., MINT 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.mint-1.16.pdf