<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dash-etal-2026-rolekv">
    <titleInfo>
        <title>RoleKV - Role-Aware KV Cache Management for the Inverted Age-Importance of LLM Agent Context</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Eashan</namePart>
        <namePart type="family">Dash</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Arun</namePart>
        <namePart type="family">Ramachandran</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Prakash</namePart>
        <namePart type="family">Raghavendra</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>When serving LLM agents in production, the KV cache grows over turns, and mainstream serving engines (e.g., vLLM) rely on least recently used (LRU) eviction to manage memory pressure. While LRU is well-understood for OS pages and single-turn LLM serving, its behavior on agent workloads remains underexplored. We present an inverted age-importance phenomenon in agent workflows, where the oldest KV blocks (system prompts, tool outputs) are the most persistently attended, while the .newest blocks (chain-of-thought reasoning) decay fastest. As a result, recency-based eviction systematically discards the wrong blocks in agent workloads, producing two distinct overheads: (1) wasted prefill work on preempt-discard engines, and (2) wasted PCIe bandwidth on CPU-offload engines. We propose RoleKV, a training-free, lossless eviction policy that evicts KV blocks based on their semantic role in agent workflows. A lightweight Role Tagger assigns each block to a semantic role, and RoleKV then evicts by role-aware decay rather than recency. Unlike model-level KV compression methods that incur quality loss, RoleKV is a system-level policy that decides where blocks reside (GPU vs. CPU/disk), without affecting model outputs. RoleKV cuts preemptions by up to 64.3%, PCIe traffic by up to 63.8%, and improves end-to-end latency by up to 1.41× on 8B-70B models, with gains that grow on longer agent traces and persist on multi-GPU setups. This recasts KV eviction in production LLM serving from a recency problem to a role-aware retention problem.</abstract>
    <identifier type="citekey">dash-etal-2026-rolekv</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1982/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>45082</start>
            <end>45103</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T RoleKV - Role-Aware KV Cache Management for the Inverted Age-Importance of LLM Agent Context
%A Dash, Eashan
%A Ramachandran, Arun
%A Raghavendra, Prakash
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F dash-etal-2026-rolekv
%X When serving LLM agents in production, the KV cache grows over turns, and mainstream serving engines (e.g., vLLM) rely on least recently used (LRU) eviction to manage memory pressure. While LRU is well-understood for OS pages and single-turn LLM serving, its behavior on agent workloads remains underexplored. We present an inverted age-importance phenomenon in agent workflows, where the oldest KV blocks (system prompts, tool outputs) are the most persistently attended, while the .newest blocks (chain-of-thought reasoning) decay fastest. As a result, recency-based eviction systematically discards the wrong blocks in agent workloads, producing two distinct overheads: (1) wasted prefill work on preempt-discard engines, and (2) wasted PCIe bandwidth on CPU-offload engines. We propose RoleKV, a training-free, lossless eviction policy that evicts KV blocks based on their semantic role in agent workflows. A lightweight Role Tagger assigns each block to a semantic role, and RoleKV then evicts by role-aware decay rather than recency. Unlike model-level KV compression methods that incur quality loss, RoleKV is a system-level policy that decides where blocks reside (GPU vs. CPU/disk), without affecting model outputs. RoleKV cuts preemptions by up to 64.3%, PCIe traffic by up to 63.8%, and improves end-to-end latency by up to 1.41× on 8B-70B models, with gains that grow on longer agent traces and persist on multi-GPU setups. This recasts KV eviction in production LLM serving from a recency problem to a role-aware retention problem.
%U https://aclanthology.org/2026.emnlp-main.1982/
%P 45082-45103

Markdown (Informal)

[RoleKV - Role-Aware KV Cache Management for the Inverted Age-Importance of LLM Agent Context](https://aclanthology.org/2026.emnlp-main.1982/) (Dash et al., EMNLP 2026)

• RoleKV - Role-Aware KV Cache Management for the Inverted Age-Importance of LLM Agent Context (Dash et al., EMNLP 2026)

ACL

• Eashan Dash, Arun Ramachandran, and Prakash Raghavendra. 2026. RoleKV - Role-Aware KV Cache Management for the Inverted Age-Importance of LLM Agent Context. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 45082–45103, Budapest, Hungary. Association for Computational Linguistics.