In on-device agentic systems, Small Language Models (SLMs) are essential for ensuring privacy-preserving personalization and low-latency execution. However, despite their capacity to handle isolated atomic tasks, during complex multi-step orchestration SLMs frequently suffer from stochastic drift—a phenomenon where minor logical deviations cascade into irrecoverable trajectory failures. In this work, we demonstrate the primacy of pre-execution planning, revealing that mitigating initial logical drift is vastly more effective than mid-trajectory interventions. Building on this insight, we propose FORST(FOundation fiRST) framework that implements sequential logical scaffolding in pre-execution planning, where predefined expert anchors are injected as fixed pre-computed KV-cache states. While the scaffolding ensures logical stability, KV-cache reuse efficiently realizes the structured anchor prefix without repeatedly prefilling static anchors. Then, we can maintain a monolithic reasoning chain within predefined logical planning boundaries and effectively eliminate the redundant computational overhead of structured anchors. Extensive evaluations on the MCPEval benchmark demonstrate that establishing this rigid logical foundation via achieves competitive or numerically higher performance than frontier models, while achieving an latency reduction on edge-device platforms.

Personal memory agents process sensitive user data, making on-device deployment with small language models (SLMs) attractive. However, existing memory-agent prompts are often monolithic specifications written for high-capacity LLMs, overloading a single inference with many requirements while leaving necessary intermediate operations implicit. This mismatch creates two bottlenecks for SLMs: prompt complexity and logical incompleteness. We introduce CAST (Compiling Agent Specifications into Scaffolded Tasks), a framework that converts a monolithic memory-agent prompt into a scaffolded execution procedure. CAST segments the original prompt into fixed, semantics-preserving main tasks, then synthesizes and places auxiliary scaffolding tasks that produce reusable intermediate artifacts; a trajectory-feedback step then adapts these scaffolds to the target SLM. At inference time, only the target SLM executes the compiled procedure. Across two memory-agent frameworks on LoCoMo and MemoryAgentBench, CAST outperforms monolithic prompting, prompt optimization, and reasoning-based baselines, while offering a stronger quality-latency trade-off than unconstrained reasoning. The results show that adapting memory agents to SLMs requires restructuring execution, not merely rewriting prompts.

Retrieval-Augmented Generation (RAG) systems are typically evaluated under static assumptions, despite being frequently corrected through user or expert feedback in deployment. Existing evaluation protocols focus on overall accuracy and fail to capture how systems adapt after feedback is introduced. We introduce feedback adaptation as a problem setting for RAG systems, which asks how effectively and how quickly corrective feedback propagates to future queries. To make this behavior measurable, we propose two evaluation axes: correction lag, which captures the delay between feedback provision and behavioral change, and post-feedback performance, which measures reliability on semantically related queries after feedback. Using these metrics, we show that training-based approaches exhibit a trade-off between delayed correction and reliable adaptation. We further propose PatchRAG, a minimal inference-time instantiation that incorporates feedback without retraining, demonstrating immediate correction and strong post-feedback generalization under the proposed evaluation. Our results highlight feedback adaptation as a previously overlooked dimension of RAG system behavior in interactive settings.

Retrieval-augmented generation (RAG) with large language models (LLMs) is especially valuable in specialized domains, where precision is critical. To more specialize the LLMs into a target domain, domain-specific RAG has recently been developed by allowing the LLM to access the target domain early via finetuning. The domain-specific RAG makes more sense in resource-constrained environments like edge devices, as they should perform a specific task (e.g. personalization) reliably using only small-scale LLMs. While the domain-specific RAG is well-aligned with edge devices in this respect, it often relies on widely-used reasoning techniques like chain-of-thought (CoT). The reasoning step is useful to understand the given external knowledge, and yet it is computationally expensive and difficult for small-scale LLMs to learn it. Tackling this, we propose the Chain of Rank (CoR) which shifts the focus from intricate lengthy reasoning to simple ranking of the reliability of input external documents. Then, CoR reduces computational complexity while maintaining high accuracy, making it particularly suited for resource-constrained environments. We attain the state-of-the-art (SOTA) results in benchmarks, and analyze its efficacy.

Effective conversational search demands a deep understanding of user intent across multiple dialogue turns. Users frequently use abbreviations and shift topics in the middle of conversations, posing challenges for conventional retrievers. While query rewriting techniques improve clarity, they often incur significant computational cost due to additional autoregressive steps. Moreover, although LLM-based retrievers demonstrate strong performance, they are not explicitly optimized to track user intent in multi-turn settings, often failing under topic drift or contextual ambiguity. To address these limitations, we propose ContextualRetriever, a novel LLM-based retriever that directly incorporates conversational context into the retrieval process. Our approach introduces: (1) a context-aware embedding mechanism that highlights the current query within the dialogue history; (2) intent-guided supervision based on high-quality rewritten queries; and (3) a training strategy that preserves the generative capabilities of the base LLM. Extensive evaluations across multiple conversational search benchmarks demonstrate that ContextualRetriever significantly outperforms existing methods while incurring no additional inference overhead.

We present CIFLEX (Contextual Instruction FLow with EXecution), a novel execution system for efficient sub-task handling in multi-turn interactions with a single on-device large language model (LLM). As LLMs become increasingly capable, a single model is expected to handle diverse sub-tasks that more effectively and comprehensively support answering user requests. Naive approach reprocesses the entire conversation context when switching between main and sub-tasks (e.g., query rewriting, summarization), incurring significant computational overhead. CIFLEX mitigates this overhead by reusing the key-value (KV) cache from the main task and injecting only task-specific instructions into isolated side paths. After sub-task execution, the model rolls back to the main path via cached context, thereby avoiding redundant prefill computation. To support sub-task selection, we also develop a hierarchical classification strategy tailored for small-scale models, decomposing multi-choice decisions into binary ones. Experiments show that CIFLEX significantly reduces computational costs without degrading task performance, enabling scalable and efficient multi-task dialogue on-device.

The customization of large language models (LLMs) for user-specified tasks gets important. However, maintaining all the customized LLMs on cloud servers incurs substantial memory and computational overheads, and uploading user data can also lead to privacy concerns. On-device LLMs can offer a promising solution by mitigating these issues. Yet, the performance of on-device LLMs is inherently constrained by the limitations of small-scaled models. To overcome these restrictions, we first propose Crayon, a novel approach for on-device LLM customization. Crayon begins by constructing a pool of diverse base adapters, and then we instantly blend them into a customized adapter without extra training. In addition, we develop a device-server hybrid inference strategy, which deftly allocates more demanding queries or non-customized tasks to a larger, more capable LLM on a server. This ensures optimal performance without sacrificing the benefits of on-device customization. We carefully craft a novel benchmark from multiple question-answer datasets, and show the efficacy of our method in the LLM customization.