Multimodal dialogue segmentation with gesture post-processing

Kodai Takahashi, Masashi Inoue


Abstract
We investigate an automatic dialogue segmentation method using both verbal and non-verbal modalities. Dialogue contents are used for the initial segmentation of dialogue; then, gesture occurrences are used to remove the incorrect segment boundaries. A unique characteristic of our method is to use verbal and non-verbal information separately. We use a three-party dialogue that is rich in gesture as data. The transcription of the dialogue is segmented into topics without prior training by using the TextTiling and U00 algorithm. Some candidates for segment boundaries - where the topic continues - are irrelevant. Those boundaries can be found and removed by locating gestures that stretch over the boundary candidates. This ltering improves the segmentation accuracy of text-only segmentation.
Anthology ID:
L14-1308
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
3433–3437
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/354_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Kodai Takahashi and Masashi Inoue. 2014. Multimodal dialogue segmentation with gesture post-processing. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 3433–3437, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Multimodal dialogue segmentation with gesture post-processing (Takahashi & Inoue, LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/354_Paper.pdf