<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="saito-etal-2026-beyond">
    <titleInfo>
        <title>Beyond Fluent Responses: Finding User Dissatisfaction in Human–AI Interactions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yuka</namePart>
        <namePart type="family">Saito</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ikumi</namePart>
        <namePart type="family">Numaya</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jun</namePart>
        <namePart type="family">Suzuki</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Reina</namePart>
        <namePart type="family">Akama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Rui</namePart>
            <namePart type="family">Sousa-Silva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henrique</namePart>
            <namePart type="family">Lopes Cardoso</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Maarit</namePart>
            <namePart type="family">Koponen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Pareja-Lora</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-471-2</identifier>
    </relatedItem>
    <abstract>As large language models generate increasingly high-quality responses, recent work on response evaluation has moved beyond response-centered criteria, such as correctness and coherence, toward user-centered criteria, including user preferences and perceived usefulness. This shift is fundamentally important for developing technologies that are truly useful to real-world users. While conventional response evaluation quantifies the quality of the response itself, users’ satisfaction in real conversations also depends on their subjective preferences and perceptions of usefulness, which do not necessarily align with response quality. This study investigates whether such user-centered signals can be detected from conversational text. Specifically, we analyze user utterances with self-reported dissatisfaction labels to examine whether user dissatisfaction in text can be detected. In English interaction, models using user utterances exceeded the 17.8% dissatisfaction label prevalence in AUPRC by up to 15 percentage points. These results suggest that user utterances reveal cues to implicit dissatisfaction. Furthermore, changes from the immediately preceding user utterance may improve dissatisfaction detection, with anomaly detection methods appearing particularly effective.</abstract>
    <identifier type="citekey">saito-etal-2026-beyond</identifier>
    <location>
        <url>https://aclanthology.org/2026.luhme-1.4/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>54</start>
            <end>69</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Beyond Fluent Responses: Finding User Dissatisfaction in Human–AI Interactions
%A Saito, Yuka
%A Numaya, Ikumi
%A Suzuki, Jun
%A Akama, Reina
%Y Sousa-Silva, Rui
%Y Lopes Cardoso, Henrique
%Y Koponen, Maarit
%Y Pareja-Lora, Antonio
%S Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-471-2
%F saito-etal-2026-beyond
%X As large language models generate increasingly high-quality responses, recent work on response evaluation has moved beyond response-centered criteria, such as correctness and coherence, toward user-centered criteria, including user preferences and perceived usefulness. This shift is fundamentally important for developing technologies that are truly useful to real-world users. While conventional response evaluation quantifies the quality of the response itself, users’ satisfaction in real conversations also depends on their subjective preferences and perceptions of usefulness, which do not necessarily align with response quality. This study investigates whether such user-centered signals can be detected from conversational text. Specifically, we analyze user utterances with self-reported dissatisfaction labels to examine whether user dissatisfaction in text can be detected. In English interaction, models using user utterances exceeded the 17.8% dissatisfaction label prevalence in AUPRC by up to 15 percentage points. These results suggest that user utterances reveal cues to implicit dissatisfaction. Furthermore, changes from the immediately preceding user utterance may improve dissatisfaction detection, with anomaly detection methods appearing particularly effective.
%U https://aclanthology.org/2026.luhme-1.4/
%P 54-69

Markdown (Informal)

[Beyond Fluent Responses: Finding User Dissatisfaction in Human–AI Interactions](https://aclanthology.org/2026.luhme-1.4/) (Saito et al., LUHME 2026)

• Beyond Fluent Responses: Finding User Dissatisfaction in Human–AI Interactions (Saito et al., LUHME 2026)

ACL

• Yuka Saito, Ikumi Numaya, Jun Suzuki, and Reina Akama. 2026. Beyond Fluent Responses: Finding User Dissatisfaction in Human–AI Interactions. In Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026), pages 54–69, Budapest, Hungary. Association for Computational Linguistics.