Large Reasoning Models (LRMs) deliver strong performance on complex reasoning tasks, but their long chain-of-thought traces incur substantial inference latency and cost. To mitigate this overhead while preserving reasoning quality, token-level routing between a small language model (SLM) and a large language model (LLM) offers one of the best efficiency-performance trade-offs. However, existing token routers face two major practical limitations: they must be retrained for every new SLM-LLM pair, and they require both models to share a tokenizer. We present UniToR, a universal token-level router that removes both constraints. UniToR predicts per-token divergence using model-agnostic signals, enabling a single router to generalize across diverse SLM-LLM pairs without retraining. To support heterogeneous model pairs, we further introduce a token-alignment mechanism that eliminates the requirement for tokenizer compatibility, enabling seamless token-level switching across models with different tokenization schemes. Experimental results show that a single UniToR router generalizes across different model families (DeepSeek-R1, Qwen3, and Phi-4), including pairs with incompatible tokenizers. UniToR matches the accuracy of R1-32B at up to 6x lower cost and successfully operates on unseen model pairs where existing token-level routing methods fail.

While retrieval-augmented generation (RAG) grounds large language models (LLMs) in external knowledge, ensuring faithful evidence utilization in complex domains like law remains challenging. LLMs often ignore retrieved facts or indiscriminately copy from noisy contexts. Existing context-aware decoding methods attempt to mitigate this by amplifying the influence of retrieved content, but they treat the context as a monolithic block—amplifying irrelevant spans alongside evidence—and intervene at every decoding step, incurring substantial inference overhead. To address this, we propose SaCoD (Selective and Confidence-aware Decoding), a training-free inference framework. SaCoD leverages the LLM’s internal attention signals to dynamically isolate salient evidence paragraphs at each step, restricting a customized copy mechanism strictly to relevant facts. Furthermore, an entropy-based gate activates this intervention only when the model is uncertain, bypassing copy operations for deterministic tokens to preserve linguistic fluency and cut computational overhead. Experiments on four legal question-answering benchmarks across three open-weight LLMs demonstrate that SaCoD significantly improves faithfulness and response correctness while substantially reducing inference latency compared with prior context-aware methods.

Speculative decoding accelerates large language model (LLM) inference through a draft-and-verify paradigm, yet existing methods face three key limitations: reliance on fixed draft templates that ignore device-specific verification costs, lack of mechanisms to assess draft token quality, and suboptimal tree expansion strategies. We introduce UniSpec, a training-free, lossless speculative decoding framework that enables robust, plug-and-play LLM acceleration across diverse hardware configurations and languages. UniSpec incorporates three novel components: (1) a device-aware calibration mechanism that determines the optimal draft size by measuring the acceptance-time trade-off on each target device; (2) a confidence score estimation module that assigns quality scores to n-grams based on the verifier’s token probabilities, enabling selective retention of high-quality draft candidates; and (3) an improved tree expansion strategy that broadens first-level exploration and applies threshold-based filtering to prune low-confidence nodes. To comprehensively evaluate multilingual performance, we create a comprehensive benchmark, covering seven languages across seven generation tasks. Experiments with various LLM architectures, hardware environments, and languages demonstrate that UniSpec consistently outperforms existing training-free methods, achieving speedups of up to 2.6x while maintaining output quality identical to standard autoregressive decoding. Our code and benchmark are publicly available.

Chain-of-thought traces drive strong LLM accuracy but dominate inference cost: output length sets both monetary cost and decoding latency. We ask which tokens in a reasoning trace actually carry the answer. A token-level analysis of the model’s own next-token distribution identifies three redundancies - discourse-element removal, tokenizer-aware abbreviation, and substitution of prose causal connectives with an arrow "X -> Y". All three leave the predictive distribution close to the original under Jensen-Shannon divergence and preserve the greedy next token in the vast majority of prefixes. We instantiate the analysis as CONCISON, a single training-free system prompt that operationalises the three rules and protects numeric, equation, and code tokens verbatim. Across 18 models from six families (2B-120B) and five reasoning benchmarks, CONCISON cuts output tokens by 24-62% on strong models at -1.1pp average accuracy, outperforms token-budget prompting in a single decoding call, and matches trained routers on the accuracy-cost frontier without any training, model pool, or extra infrastructure.

Large Language Models (LLMs) demonstrate strong multilingual capabilities but are costly to deploy due to their size and computational demands. To mitigate this, compression techniques such as pruning and quantization are widely used. However, these methods face two key limitations: (1) they assume access to high-quality instruction or calibration data, which is often unavailable for low-resource languages; and (2) they aim to preserve multilingual generality, making them inefficient for language-specific applications. We introduce LangCompress, a language-aware compression framework that enhances existing compression methods for targeted deployment. LangCompress is method-agnostic and improves state-of-the-art pruning and quantization approaches. It features two core components: an iterative self-supervised pipeline for generating instruction data in the target language, and a vocabulary simplification strategy that reduces the LM head to focus on key tokens. Experiments on perplexity, translation, and summarization tasks show that LangCompress improves performance in the target language. The code and data are publicly available.

The growing volume of scientific literature in polymer science presents a significant challenge for researchers attempting to extract and annotate domain-specific entities, such as polymer names, material properties, and related information. Manual annotation of these documents is both time-consuming and prone to error due to the complexity of scientific language. To address this, we introduce PolyMinder, an automated support system designed to assist polymer scientists in extracting and annotating polymer-related entities and their relationships from scientific documents. The system utilizes recent advanced Named Entity Recognition (NER) and Relation Extraction (RE) models tailored to the polymer domain. PolyMinder streamlines the annotation process by providing a web-based interface where users can visualize, verify, and refine the extracted information before finalizing the annotations. The system’s source code is made publicly available to facilitate further research and development in this field. Our system can be accessed through the following URL: https://www.jaist.ac.jp/is/labs/nguyen-lab/systems/polyminder

Inference with modern Large Language Models (LLMs) is both computationally expensive and time-consuming. Speculative decoding has emerged as a promising solution, but existing approaches face key limitations: training-based methods require a draft model that is challenging to obtain and lacks generalizability, while training-free methods offer limited speedup gains. In this work, we present Spectra, a novel framework for accelerating LLM inference without the need for additional training or modification to the original LLM. Spectra introduces two new techniques for efficiently utilizing internal and external speculation, each outperforming corresponding state-of-the-art (SOTA) methods independently. When combined, these techniques achieve up to a 4.08x speedup across various benchmarks and LLM architectures, significantly surpassing existing training-free approaches. The implementation of Spectra is publicly available.