Ryohei Kamei
Author directory2026
What Counts as Hallucination? A Scoping Review of Tasks, Symptoms, and Convergence Expectations in LLM Research
Toshiki Kuramoto | Ryohei Kamei | Jun Suzuki
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Toshiki Kuramoto | Ryohei Kamei | Jun Suzuki
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Hallucination in large language models (LLMs) and related generative models has become a central issue for reliability and practical deployment. Yet prior work does not consistently define what is treated as hallucination or the level of expectation under which model outputs are evaluated. This study conducts a scoping review of 244 papers published between 2018 and 2025, analyzing definitions of hallucination, output-side symptoms, and Convergence Expectation (CE). The results show that approximately 90% of the papers provide some form of operational definition, but the symptoms targeted by these operationalizations vary substantially by task. "Addition" appears across many tasks as a common symptom, whereas "Transformation" and "Deviation" strongly depend on task structure and input constraints. Analyses by CE band further show that, even within the same task, the symptoms problematized as hallucination vary according to the strictness of expected output convergence. These findings indicate that hallucination is not a fixed failure inherent in model outputs, but a phenomenon shaped by task, input constraints, evaluation criteria, and the expectations of humans or evaluators. This study reframes hallucination as expectation mismatch and argues that definitions, symptom scope, and expectation levels must be made explicit when comparing research findings and designing evaluations.
2024
A Multimodal Dialogue System to Lead Consensus Building with Emotion-Displaying
Shinnosuke Nozue | Yuto Nakano | Shoji Moriya | Tomoki Ariyama | Kazuma Kokuta | Suchun Xie | Kai Sato | Shusaku Sone | Ryohei Kamei | Reina Akama | Yuichiroh Matsubayashi | Keisuke Sakaguchi
Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Shinnosuke Nozue | Yuto Nakano | Shoji Moriya | Tomoki Ariyama | Kazuma Kokuta | Suchun Xie | Kai Sato | Shusaku Sone | Ryohei Kamei | Reina Akama | Yuichiroh Matsubayashi | Keisuke Sakaguchi
Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue
The evolution of large language models has enabled fluent dialogue, increasing interest in the coexistence of humans and avatars. An essential aspect of achieving this coexistence involves developing sophisticated dialogue systems that can influence user behavior. In this background, we propose an effective multimodal dialogue system designed to promote consensus building with humans. Our system employs a slot-filling strategy to guide discussions and attempts to influence users with suggestions through emotional expression and intent conveyance via its avatar. These innovations have resulted in our system achieving the highest performance in a competition evaluating consensus building between humans and dialogue systems. We hope that our research will promote further discussion on the development of dialogue systems that enhance consensus building in human collaboration.
Detecting Response Generation Not Requiring Factual Judgment
Ryohei Kamei | Daiki Shiono | Reina Akama | Jun Suzuki
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)
Ryohei Kamei | Daiki Shiono | Reina Akama | Jun Suzuki
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)
With the remarkable development of large language models (LLMs), ensuring the factuality of output has become a challenge.However, having all the contents of the response with given knowledge or facts is not necessarily a good thing in dialogues.This study aimed to achieve both attractiveness and factuality in a dialogue response for which a task was set to predict sentences that do not require factual correctness judgment such as agreeing, or personal opinions/feelings.We created a dataset, dialogue dataset annotated with fact-check-needed label (DDFC), for this task via crowdsourcing, and classification tasks were performed on several models using this dataset.The model with the highest classification accuracy could yield about 88% accurate classification results.