When the key-value (KV) cache fills, a serving engine must choose which request to evict. We present PreemptBench, comparing seven preemption-target policies across seven reasoning-model families and two benchmarks. Our service-level objective (SLO) breach rate counts every accepted request, including late completions, post-acceptance failures, and unfinished work; this survivorship correction prevents deferred backlog from being omitted from the metric. In the low-breach regime, where last-in, first-out (LIFO) breach is below 15%, young-target-biased policies form a lower-breach cluster than policies without this bias, including random and largest-KV, consistent with a stable incumbent cohort; graded eviction depth is secondary. Within a pure remaining-decode rule, replacing the online predictor with a privileged empirical statistical length oracle, constructed from prior temperature-zero runs, yields no significant gain; neither significantly outperforms LIFO. It summarizes a prompt-conditioned length distribution, not the current execution’s realization, and is not SLO-optimal. Finally, the uncensored single-stream completion-time tail defines a conceptual feasibility floor, but our traces provide only a censored low-load proxy. Post-hoc rescoring across 600–1800-second deadlines finds aggregate memory-freeing reversals at tighter SLOs, but none at 1800 seconds. Thus LIFO is a strong conditional baseline, not a universal optimum.

Subword tokenization is a commonly used input pre-processing step in most recent NLP models. However, it limits the models’ ability to leverage end-to-end task learning. Its frequency-based vocabulary creation compromises tokenization in low-resource languages, leading models to produce suboptimal representations. Additionally, the dependency on a fixed vocabulary limits the subword models’ adaptability across languages and domains. In this work, we propose a vocabulary-free neural tokenizer by distilling segmentation information from heuristic-based subword tokenization. We pre-train our character-based tokenizer by processing unique words from multilingual corpus, thereby extensively increasing word diversity across languages. Unlike the predefined and fixed vocabularies in subword methods, our tokenizer allows end-to-end task learning, resulting in optimal task-specific tokenization. The experimental results show that replacing the subword tokenizer with our neural tokenizer consistently improves performance on multilingual (NLI) and code-switching (sentiment analysis) tasks, with larger gains in low-resource languages. Additionally, our neural tokenizer exhibits a robust performance on downstream tasks when adversarial noise is present (typos and misspelling), further increasing the initial improvements over statistical subword tokenizers.

Self-learning paradigms in large-scale conversational AI agents tend to leverage user feedback in bridging between what they say and what they mean. However, such learning, particularly in Markov-based query rewriting systems have far from addressed the impact of these models on future training where successive feedback is inevitably contingent on the rewrite itself, especially in a continually updating environment. In this paper, we explore the consequences of this inherent lack of self-awareness towards impairing the model performance, ultimately resulting in both Type I and II errors over time. To that end, we propose augmenting the Markov Graph construction with a superposition-based adjacency matrix. Here, our method leverages an induced stochasticity to reactively learn a locally-adaptive decision boundary based on the performance of the individual rewrites in a bi-variate beta setting. We also surface a data augmentation strategy that leverages template-based generation in abridging complex conversation hierarchies of dialogs so as to simplify the learning process. All in all, we demonstrate that our self-aware model improves the overall PR-AUC by 27.45%, achieves a relative defect reduction of up to 31.22%, and is able to adapt quicker to changes in global preferences across a large number of customers.