<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kireyev-2026-raw">
    <titleInfo>
        <title>RAW: Refusal Aversion in the Wild, A Causal Measurement Method for Deployed LLMs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Pavel</namePart>
        <namePart type="family">Kireyev</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Pepa</namePart>
            <namePart type="family">Atanasova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gholamreza</namePart>
            <namePart type="family">Haffari</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Minghan</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-460-6</identifier>
    </relatedItem>
    <abstract>Deployed LLMs field hundreds of millions of conversations a day and refuse a non-trivial fraction of user requests, yet the user-side cost of refusal, whether users disengage and by how much, is unmeasured on any production model: a blind spot for the consumer-facing platforms built on them. A/B-testing refusal is ethically and operationally constrained on frontier-lab products and unavailable to third-party platforms deploying these models through a fixed API. We introduce RAW, a method that uses LLM stochasticity as a natural experiment: at near-identical prompts in any existing log, the same model sometimes refuses and sometimes does not, and that variation identifies the causal user-side effect of refusal. Applying the method to 15 deployed LLMs in LMSYS-Chat-1M and WildChat, with user-clustered inference, we find that for about half of the models we can measure precisely, a refusal costs 3 to 17 pp of subsequent re-engagement; the rest show no distinguishable effect, among them one deployment whose positive point estimate we trace to persistent users retrying the refused request. Our contribution is the measurement method, which runs on any deployed model from logs alone for about 0.30 per 1,000 conversations. We release labels, moderation scores, prompt embeddings, and pipeline code.</abstract>
    <identifier type="citekey">kireyev-2026-raw</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-industry.55/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>902</start>
            <end>927</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T RAW: Refusal Aversion in the Wild, A Causal Measurement Method for Deployed LLMs
%A Kireyev, Pavel
%Y Atanasova, Pepa
%Y Haffari, Gholamreza
%Y Wang, Minghan
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-460-6
%F kireyev-2026-raw
%X Deployed LLMs field hundreds of millions of conversations a day and refuse a non-trivial fraction of user requests, yet the user-side cost of refusal, whether users disengage and by how much, is unmeasured on any production model: a blind spot for the consumer-facing platforms built on them. A/B-testing refusal is ethically and operationally constrained on frontier-lab products and unavailable to third-party platforms deploying these models through a fixed API. We introduce RAW, a method that uses LLM stochasticity as a natural experiment: at near-identical prompts in any existing log, the same model sometimes refuses and sometimes does not, and that variation identifies the causal user-side effect of refusal. Applying the method to 15 deployed LLMs in LMSYS-Chat-1M and WildChat, with user-clustered inference, we find that for about half of the models we can measure precisely, a refusal costs 3 to 17 pp of subsequent re-engagement; the rest show no distinguishable effect, among them one deployment whose positive point estimate we trace to persistent users retrying the refused request. Our contribution is the measurement method, which runs on any deployed model from logs alone for about 0.30 per 1,000 conversations. We release labels, moderation scores, prompt embeddings, and pipeline code.
%U https://aclanthology.org/2026.emnlp-industry.55/
%P 902-927

Markdown (Informal)

[RAW: Refusal Aversion in the Wild, A Causal Measurement Method for Deployed LLMs](https://aclanthology.org/2026.emnlp-industry.55/) (Kireyev, EMNLP 2026)

• RAW: Refusal Aversion in the Wild, A Causal Measurement Method for Deployed LLMs (Kireyev, EMNLP 2026)

ACL

• Pavel Kireyev. 2026. RAW: Refusal Aversion in the Wild, A Causal Measurement Method for Deployed LLMs. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 902–927, Budapest, Hungary. Association for Computational Linguistics.