MONAH: Multi-Modal Narratives for Humans to analyze conversations

Joshua Y. Kim, Kalina Yacef, Greyson Kim, Chunfeng Liu, Rafael Calvo, Silas Taylor


Abstract
In conversational analyses, humans manually weave multimodal information into the transcripts, which is significantly time-consuming. We introduce a system that automatically expands the verbatim transcripts of video-recorded conversations using multimodal data streams. This system uses a set of preprocessing rules to weave multimodal annotations into the verbatim transcripts and promote interpretability. Our feature engineering contributions are two-fold: firstly, we identify the range of multimodal features relevant to detect rapport-building; secondly, we expand the range of multimodal annotations and show that the expansion leads to statistically significant improvements in detecting rapport-building.
Anthology ID:
2021.eacl-main.37
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
466–479
Language:
URL:
https://aclanthology.org/2021.eacl-main.37
DOI:
10.18653/v1/2021.eacl-main.37
Bibkey:
Cite (ACL):
Joshua Y. Kim, Kalina Yacef, Greyson Kim, Chunfeng Liu, Rafael Calvo, and Silas Taylor. 2021. MONAH: Multi-Modal Narratives for Humans to analyze conversations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 466–479, Online. Association for Computational Linguistics.
Cite (Informal):
MONAH: Multi-Modal Narratives for Humans to analyze conversations (Kim et al., EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.37.pdf
Code
 SpectData/MONAH