<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="qian-he-2026-llm">
    <titleInfo>
        <title>In LLM Reasoning, there is Irrationality on top of Value Misalignment</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kejiang</namePart>
        <namePart type="family">Qian</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fengxiang</namePart>
        <namePart type="family">He</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Significant progress has been made in aligning LLMs with target value functions. We argue that, even when an LLM has been well aligned in (post-)training, it may still fail to maximise the aligned value in reasoning. We mathematically formalise this gap as rational value risk: the utility discrepancy between a model’s deployed reasoning strategy and its rational counterpart whose responses maximise utility in the steepest direction. The estimation error of rational value risk is further decomposed into three components from bounded prompts, bounded responses, and imperfect verifiers. Extensive experiments are conducted, covering models Llama-3.1, Qwen-2.5, Tülu-3 families (7B-72B), GPT-5.2, GPT-5.5, and DeepSeek-V4, and benchmarks UltraFeedback, AlpacaEval, GSM8K, MATH, HumanEval, and MathArena. The results validate that (1) rational value risk is widespread; (2) value alignment can reduce, but cannot avoid, it; (3) self-consistency can improve rationality; and (4) a longer chain of thought improves rationality but with diminishing returns. The code is at https://github.com/EVIEHub/LLM-Rationality.</abstract>
    <identifier type="citekey">qian-he-2026-llm</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.2249/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>51157</start>
            <end>51178</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T In LLM Reasoning, there is Irrationality on top of Value Misalignment
%A Qian, Kejiang
%A He, Fengxiang
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F qian-he-2026-llm
%X Significant progress has been made in aligning LLMs with target value functions. We argue that, even when an LLM has been well aligned in (post-)training, it may still fail to maximise the aligned value in reasoning. We mathematically formalise this gap as rational value risk: the utility discrepancy between a model’s deployed reasoning strategy and its rational counterpart whose responses maximise utility in the steepest direction. The estimation error of rational value risk is further decomposed into three components from bounded prompts, bounded responses, and imperfect verifiers. Extensive experiments are conducted, covering models Llama-3.1, Qwen-2.5, Tülu-3 families (7B-72B), GPT-5.2, GPT-5.5, and DeepSeek-V4, and benchmarks UltraFeedback, AlpacaEval, GSM8K, MATH, HumanEval, and MathArena. The results validate that (1) rational value risk is widespread; (2) value alignment can reduce, but cannot avoid, it; (3) self-consistency can improve rationality; and (4) a longer chain of thought improves rationality but with diminishing returns. The code is at https://github.com/EVIEHub/LLM-Rationality.
%U https://aclanthology.org/2026.emnlp-main.2249/
%P 51157-51178

Markdown (Informal)

[In LLM Reasoning, there is Irrationality on top of Value Misalignment](https://aclanthology.org/2026.emnlp-main.2249/) (Qian & He, EMNLP 2026)

• In LLM Reasoning, there is Irrationality on top of Value Misalignment (Qian & He, EMNLP 2026)

ACL

• Kejiang Qian and Fengxiang He. 2026. In LLM Reasoning, there is Irrationality on top of Value Misalignment. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 51157–51178, Budapest, Hungary. Association for Computational Linguistics.