<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jin-etal-2024-automatic">
    <titleInfo>
        <title>Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Claire</namePart>
        <namePart type="family">Jin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sudha</namePart>
        <namePart type="family">Rao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xiangyu</namePart>
        <namePart type="family">Peng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Portia</namePart>
        <namePart type="family">Botchway</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jessica</namePart>
        <namePart type="family">Quaye</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chris</namePart>
        <namePart type="family">Brockett</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Bill</namePart>
        <namePart type="family">Dolan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Advancements in large language models (LLMs) are revolutionizing interactive game design, enabling dynamic plotlines and interactions between players and non-player characters (NPCs). However, LLMs may exhibit flaws such as hallucinations, forgetfulness, or misinterpretations of prompts, causing logical inconsistencies and unexpected deviations from intended designs. Automated techniques for detecting such game bugs are still lacking. To address this, we propose a systematic LLM-based method for automatically identifying such bugs from player game logs, eliminating the need for collecting additional data such as post-play surveys. Applied to a text-based game DejaBoom!, our approach effectively identifies bugs inherent in LLM-powered interactive games, surpassing unstructured LLM-powered bug-catching methods and filling the gap in automated detection of logical and design flaws.</abstract>
    <identifier type="citekey">jin-etal-2024-automatic</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-acl.907</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.907</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>15353</start>
            <end>15368</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs
%A Jin, Claire
%A Rao, Sudha
%A Peng, Xiangyu
%A Botchway, Portia
%A Quaye, Jessica
%A Brockett, Chris
%A Dolan, Bill
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics: ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F jin-etal-2024-automatic
%X Advancements in large language models (LLMs) are revolutionizing interactive game design, enabling dynamic plotlines and interactions between players and non-player characters (NPCs). However, LLMs may exhibit flaws such as hallucinations, forgetfulness, or misinterpretations of prompts, causing logical inconsistencies and unexpected deviations from intended designs. Automated techniques for detecting such game bugs are still lacking. To address this, we propose a systematic LLM-based method for automatically identifying such bugs from player game logs, eliminating the need for collecting additional data such as post-play surveys. Applied to a text-based game DejaBoom!, our approach effectively identifies bugs inherent in LLM-powered interactive games, surpassing unstructured LLM-powered bug-catching methods and filling the gap in automated detection of logical and design flaws.
%R 10.18653/v1/2024.findings-acl.907
%U https://aclanthology.org/2024.findings-acl.907
%U https://doi.org/10.18653/v1/2024.findings-acl.907
%P 15353-15368

Markdown (Informal)

[Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs](https://aclanthology.org/2024.findings-acl.907) (Jin et al., Findings 2024)

• Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs (Jin et al., Findings 2024)

ACL

• Claire Jin, Sudha Rao, Xiangyu Peng, Portia Botchway, Jessica Quaye, Chris Brockett, and Bill Dolan. 2024. Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15353–15368, Bangkok, Thailand. Association for Computational Linguistics.