<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kim-yang-2026-diagnosing">
    <titleInfo>
        <title>Diagnosing Prompt Sensitivity and Interaction Instability in LLM-Based Humor Judgment</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Dohyeon</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jihoon</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruifeng</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ayu</namePart>
            <namePart type="family">Purwarianti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tong</namePart>
            <namePart type="family">Rong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun</namePart>
            <namePart type="family">Suzuki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-439-2</identifier>
    </relatedItem>
    <abstract>Humor judgment is subjective and often depends on ambiguity, expectation violation, and contextual reinterpretation, so the reliability of an LLM judge cannot be assessed from final detection performance alone. Existing LLM-based humor research and LLM-as-a-Judge studies largely focus on final performance, generation quality, or agreement with human evaluation, leaving prompt sensitivity and rebuttal robustness in subjective judgment tasks relatively underexplored. This paper analyzes how prompt design, structured interpretation, and user rebuttals shape LLM-based humor judgment. In zero-shot sentence-only judgments, models with similar F1 scores show substantially different FPR–FNR values, shifting between over-accepting and under-predicting humor depending on prompt formulation. Few-shot examples partially reduce these directional fluctuations, but their effects remain model-dependent. We further propose the Incongruity-Resolution Judgment framework, which compares a humor-supporting interpretation with a counter interpretation before producing a final judgment. Rather than uniformly improving performance, this framework serves as a diagnostic structure for examining how competing interpretations reshape model judgment behavior. Finally, explicit user rebuttals show that high initial performance does not guarantee robustness, highlighting the need to evaluate judgment persistence, directional error patterns, and interaction-level judgment trajectories.</abstract>
    <identifier type="citekey">kim-yang-2026-diagnosing</identifier>
    <location>
        <url>https://aclanthology.org/2026.aacl-long.126/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>2756</start>
            <end>2798</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Diagnosing Prompt Sensitivity and Interaction Instability in LLM-Based Humor Judgment
%A Kim, Dohyeon
%A Yang, Jihoon
%Y Zhang, Yue
%Y Xu, Ruifeng
%Y Purwarianti, Ayu
%Y Rong, Tong
%Y Suzuki, Jun
%S Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-439-2
%F kim-yang-2026-diagnosing
%X Humor judgment is subjective and often depends on ambiguity, expectation violation, and contextual reinterpretation, so the reliability of an LLM judge cannot be assessed from final detection performance alone. Existing LLM-based humor research and LLM-as-a-Judge studies largely focus on final performance, generation quality, or agreement with human evaluation, leaving prompt sensitivity and rebuttal robustness in subjective judgment tasks relatively underexplored. This paper analyzes how prompt design, structured interpretation, and user rebuttals shape LLM-based humor judgment. In zero-shot sentence-only judgments, models with similar F1 scores show substantially different FPR–FNR values, shifting between over-accepting and under-predicting humor depending on prompt formulation. Few-shot examples partially reduce these directional fluctuations, but their effects remain model-dependent. We further propose the Incongruity-Resolution Judgment framework, which compares a humor-supporting interpretation with a counter interpretation before producing a final judgment. Rather than uniformly improving performance, this framework serves as a diagnostic structure for examining how competing interpretations reshape model judgment behavior. Finally, explicit user rebuttals show that high initial performance does not guarantee robustness, highlighting the need to evaluate judgment persistence, directional error patterns, and interaction-level judgment trajectories.
%U https://aclanthology.org/2026.aacl-long.126/
%P 2756-2798

Markdown (Informal)

[Diagnosing Prompt Sensitivity and Interaction Instability in LLM-Based Humor Judgment](https://aclanthology.org/2026.aacl-long.126/) (Kim & Yang, AACL-IJCNLP 2026)

• Diagnosing Prompt Sensitivity and Interaction Instability in LLM-Based Humor Judgment (Kim & Yang, AACL-IJCNLP 2026)

ACL

• Dohyeon Kim and Jihoon Yang. 2026. Diagnosing Prompt Sensitivity and Interaction Instability in LLM-Based Humor Judgment. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2756–2798, Hengqin, China. Association for Computational Linguistics.