Hirofumi Kikuchi


2025

pdf bib
DSLCMM: A Multimodal Human-Machine Dialogue Corpus Built through Competitions
Ryuichiro Higashinaka | Tetsuro Takahashi | Shinya Iizuka | Sota Horiuchi | Michimasa Inaba | Zhiyang Qi | Yuta Sasaki | Kotaro Funakoshi | Shoji Moriya | Shiki Sato | Takashi Minato | Kurima Sakai | Tomo Funayama | Masato Komuro | Hiroyuki Nishikawa | Ryosaku Makino | Hirofumi Kikuchi | Mayumi Usami
Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology

A corpus of dialogues between multimodal systems and humans is indispensable for the development and improvement of such systems. However, there is a shortage of human-machine multimodal dialogue datasets, which hinders the widespread deployment of these systems in society. To address this issue, we construct a Japanese multimodal human-machine dialogue corpus, DSLCMM, by collecting and organizing data from the Dialogue System Live Competitions (DSLCs). This paper details the procedure for constructing the corpus and presents our analysis of the relationship between various dialogue features and evaluation scores provided by users.

pdf bib
Analyzing Dialogue System Behavior in a Specific Situation Requiring Interpersonal Consideration
Tetsuro Takahashi | Hirofumi Kikuchi | Jie Yang | Hiroyuki Nishikawa | Masato Komuro | Ryosaku Makino | Shiki Sato | Yuta Sasaki | Shinji Iwata | Asahi Hentona | Takato Yamazaki | Shoji Moriya | Masaya Ohagi | Zhiyang Qi | Takashi Kodama | Akinobu Lee | Takashi Minato | Kurima Sakai | Tomo Funayama | Kotaro Funakoshi | Mayumi Usami | Michimasa Inaba | Ryuichiro Higashinaka
Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue

In human-human conversation, interpersonal consideration for the interlocutor is essential, and similar expectations are increasingly placed on dialogue systems. This study examines the behavior of dialogue systems in a specific interpersonal scenario where a user vents frustrations and seeks emotional support from a long-time friend represented by a dialogue system. We conducted a human evaluation and qualitative analysis of 15 dialogue systems under this setting. These systems implemented diverse strategies, such as structuring dialogue into distinct phases, modeling interpersonal relationships, and incorporating cognitive behavioral therapy techniques. Our analysis reveals that these approaches contributed to improved perceived empathy, coherence, and appropriateness, highlighting the importance of design choices in socially sensitive dialogue.

pdf bib
Key Challenges in Multimodal Task-Oriented Dialogue Systems: Insights from a Large Competition-Based Dataset
Shiki Sato | Shinji Iwata | Asahi Hentona | Yuta Sasaki | Takato Yamazaki | Shoji Moriya | Masaya Ohagi | Hirofumi Kikuchi | Jie Yang | Zhiyang Qi | Takashi Kodama | Akinobu Lee | Masato Komuro | Hiroyuki Nishikawa | Ryosaku Makino | Takashi Minato | Kurima Sakai | Tomo Funayama | Kotaro Funakoshi | Mayumi Usami | Michimasa Inaba | Tetsuro Takahashi | Ryuichiro Higashinaka
Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Challenges in multimodal task-oriented dialogue between humans and systems, particularly those involving audio and visual interactions, have not been sufficiently explored or shared, forcing researchers to define improvement directions individually without a clearly shared roadmap. To address these challenges, we organized a competition for multimodal task-oriented dialogue systems and constructed a large competition-based dataset of 1,865 minutes of Japanese task-oriented dialogues. This dataset includes audio and visual interactions between diverse systems and human participants. After analyzing system behaviors identified as problematic by the human participants in questionnaire surveys and notable methods employed by the participating teams, we identified key challenges in multimodal task-oriented dialogue systems and discussed potential directions for overcoming these challenges.