@inproceedings{kano-etal-2023-aiwolfdial,
title = "{AIW}olf{D}ial 2023: Summary of Natural Language Division of 5th International {AIW}olf Contest",
author = "Kano, Yoshinobu and
Watanabe, Neo and
Kagaminuma, Kaito and
Aranha, Claus and
Lee, Jaewon and
Hauer, Benedek and
Shibata, Hisaichi and
Miki, Soichiro and
Nakamura, Yuta and
Okubo, Takuya and
Shigemura, Soga and
Ito, Rei and
Takashima, Kazuki and
Fukuda, Tomoki and
Wakutani, Masahiro and
Hatanaka, Tomoya and
Uchida, Mami and
Abe, Mikio and
Mikami, Akihiro and
Otsuki, Takashi and
Qi, Zhiyang and
Harada, Kei and
Inaba, Michimasa and
Katagami, Daisuke and
Osawa, Hirotaka and
Toriumi, Fujio",
editor = "Mille, Simon",
booktitle = "Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges",
month = sep,
year = "2023",
address = "Prague, Czechia",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.inlg-genchal.13",
pages = "84--100",
abstract = "We held our 5th annual AIWolf international contest to automatically play the Werewolf game {``}Mafia{''}, where players try finding liars via conversations, aiming at promoting developments in creating agents of more natural conversations in higher level, such as longer contexts, personal relationships, semantics, pragmatics, and logics, revealing the capabilities and limits of the generative AIs. In our Natural Language Division of the contest, we had six Japanese speaking agents from five teams, and three English speaking agents, to mutually run games. By using the game logs, We performed human subjective evaluations and detailed log analysis. We found that the entire system performance has largely improved over the previous year, due to the recent advantages of the LLMs. However, it is not perfect at all yet; the generated talks are sometimes inconsistent with the game actions, it is still doubtful that the agents could infer roles by logics rather than superficial utterance generations. It is not explicitly observed in this log but it would be still difficult to make an agent telling a lie, pretend as a villager but it has an opposite goal inside. Our future work includes to reveal the capability of the LLMs, whether they can make the duality of the {``}liar{''}, in other words, holding a {``}true{''} and a {``}false{''} circumstances of the agent at the same time, even holding what these circumstances look like from other agents.",
}
<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kano-etal-2023-aiwolfdial">
<titleInfo>
<title>AIWolfDial 2023: Summary of Natural Language Division of 5th International AIWolf Contest</title>
</titleInfo>
<name type="personal">
<namePart type="given">Yoshinobu</namePart>
<namePart type="family">Kano</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Neo</namePart>
<namePart type="family">Watanabe</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Kaito</namePart>
<namePart type="family">Kagaminuma</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Claus</namePart>
<namePart type="family">Aranha</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jaewon</namePart>
<namePart type="family">Lee</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Benedek</namePart>
<namePart type="family">Hauer</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hisaichi</namePart>
<namePart type="family">Shibata</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Soichiro</namePart>
<namePart type="family">Miki</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yuta</namePart>
<namePart type="family">Nakamura</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Takuya</namePart>
<namePart type="family">Okubo</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Soga</namePart>
<namePart type="family">Shigemura</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Rei</namePart>
<namePart type="family">Ito</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Kazuki</namePart>
<namePart type="family">Takashima</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Tomoki</namePart>
<namePart type="family">Fukuda</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Masahiro</namePart>
<namePart type="family">Wakutani</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Tomoya</namePart>
<namePart type="family">Hatanaka</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Mami</namePart>
<namePart type="family">Uchida</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Mikio</namePart>
<namePart type="family">Abe</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Akihiro</namePart>
<namePart type="family">Mikami</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Takashi</namePart>
<namePart type="family">Otsuki</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Zhiyang</namePart>
<namePart type="family">Qi</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Kei</namePart>
<namePart type="family">Harada</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Michimasa</namePart>
<namePart type="family">Inaba</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Daisuke</namePart>
<namePart type="family">Katagami</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hirotaka</namePart>
<namePart type="family">Osawa</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Fujio</namePart>
<namePart type="family">Toriumi</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2023-09</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges</title>
</titleInfo>
<name type="personal">
<namePart type="given">Simon</namePart>
<namePart type="family">Mille</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Prague, Czechia</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
</relatedItem>
<abstract>We held our 5th annual AIWolf international contest to automatically play the Werewolf game “Mafia”, where players try finding liars via conversations, aiming at promoting developments in creating agents of more natural conversations in higher level, such as longer contexts, personal relationships, semantics, pragmatics, and logics, revealing the capabilities and limits of the generative AIs. In our Natural Language Division of the contest, we had six Japanese speaking agents from five teams, and three English speaking agents, to mutually run games. By using the game logs, We performed human subjective evaluations and detailed log analysis. We found that the entire system performance has largely improved over the previous year, due to the recent advantages of the LLMs. However, it is not perfect at all yet; the generated talks are sometimes inconsistent with the game actions, it is still doubtful that the agents could infer roles by logics rather than superficial utterance generations. It is not explicitly observed in this log but it would be still difficult to make an agent telling a lie, pretend as a villager but it has an opposite goal inside. Our future work includes to reveal the capability of the LLMs, whether they can make the duality of the “liar”, in other words, holding a “true” and a “false” circumstances of the agent at the same time, even holding what these circumstances look like from other agents.</abstract>
<identifier type="citekey">kano-etal-2023-aiwolfdial</identifier>
<location>
<url>https://aclanthology.org/2023.inlg-genchal.13</url>
</location>
<part>
<date>2023-09</date>
<extent unit="page">
<start>84</start>
<end>100</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T AIWolfDial 2023: Summary of Natural Language Division of 5th International AIWolf Contest
%A Kano, Yoshinobu
%A Watanabe, Neo
%A Kagaminuma, Kaito
%A Aranha, Claus
%A Lee, Jaewon
%A Hauer, Benedek
%A Shibata, Hisaichi
%A Miki, Soichiro
%A Nakamura, Yuta
%A Okubo, Takuya
%A Shigemura, Soga
%A Ito, Rei
%A Takashima, Kazuki
%A Fukuda, Tomoki
%A Wakutani, Masahiro
%A Hatanaka, Tomoya
%A Uchida, Mami
%A Abe, Mikio
%A Mikami, Akihiro
%A Otsuki, Takashi
%A Qi, Zhiyang
%A Harada, Kei
%A Inaba, Michimasa
%A Katagami, Daisuke
%A Osawa, Hirotaka
%A Toriumi, Fujio
%Y Mille, Simon
%S Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges
%D 2023
%8 September
%I Association for Computational Linguistics
%C Prague, Czechia
%F kano-etal-2023-aiwolfdial
%X We held our 5th annual AIWolf international contest to automatically play the Werewolf game “Mafia”, where players try finding liars via conversations, aiming at promoting developments in creating agents of more natural conversations in higher level, such as longer contexts, personal relationships, semantics, pragmatics, and logics, revealing the capabilities and limits of the generative AIs. In our Natural Language Division of the contest, we had six Japanese speaking agents from five teams, and three English speaking agents, to mutually run games. By using the game logs, We performed human subjective evaluations and detailed log analysis. We found that the entire system performance has largely improved over the previous year, due to the recent advantages of the LLMs. However, it is not perfect at all yet; the generated talks are sometimes inconsistent with the game actions, it is still doubtful that the agents could infer roles by logics rather than superficial utterance generations. It is not explicitly observed in this log but it would be still difficult to make an agent telling a lie, pretend as a villager but it has an opposite goal inside. Our future work includes to reveal the capability of the LLMs, whether they can make the duality of the “liar”, in other words, holding a “true” and a “false” circumstances of the agent at the same time, even holding what these circumstances look like from other agents.
%U https://aclanthology.org/2023.inlg-genchal.13
%P 84-100
Markdown (Informal)
[AIWolfDial 2023: Summary of Natural Language Division of 5th International AIWolf Contest](https://aclanthology.org/2023.inlg-genchal.13) (Kano et al., INLG-SIGDIAL 2023)
ACL
- Yoshinobu Kano, Neo Watanabe, Kaito Kagaminuma, Claus Aranha, Jaewon Lee, Benedek Hauer, Hisaichi Shibata, Soichiro Miki, Yuta Nakamura, Takuya Okubo, Soga Shigemura, Rei Ito, Kazuki Takashima, Tomoki Fukuda, Masahiro Wakutani, Tomoya Hatanaka, Mami Uchida, Mikio Abe, Akihiro Mikami, et al.. 2023. AIWolfDial 2023: Summary of Natural Language Division of 5th International AIWolf Contest. In Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges, pages 84–100, Prague, Czechia. Association for Computational Linguistics.