Given a Video Large Language Model (VLLM) and a pre-recorded video, how can we minimize the cumulative inference latency over future questions? Addressing this is crucial since VLLMs not only serve a massive volume of questions over the video’s lifespan but also incur substantial latency for each question. Existing methods compress visual tokens generated from a video to accelerate inference for each question. However, under aggressive compression ratios, prior approaches suffer from severe accuracy degradation, still resulting in high inference latency on each question. In this paper, we propose Dive, a visual KV cache compression method that preserves accuracy even under aggressive compression ratios. To maintain accuracy, Dive conserves the temporal context of the input video by evicting KV pairs after the prefill stage, rather than filtering out visual tokens before the VLLM processes the tokens through attention mechanisms. Dive significantly reduces per-question decoding latency by carefully evicting KV pairs, thereby achieving low cumulative latency over a massive volume of questions. Experiments show that Dive retains 98.2% of accuracy, using only 5% of the KV cache in Qwen2.5-VL.

How can we accurately compress a sparse mixture-of-experts (SMoE) model? SMoE models are widely adopted in modern LLMs for reducing activated computation, but their deployment remains memory-intensive due to extensive expert parameters. To address this bottleneck, expert compression reduces redundant expert parameters. However, previous methods decrease the number of experts involved in token-dependent routing, inducing output deviations from the original model. Expert decomposition is a promising alternative, but existing approaches suffer from three major limitations: a single shared base, component memory overhead, and suboptimal objectives. In this work, we introduce five desired properties for accurate expert compression. Then, we propose ResCode, an accurate expert decomposition method based on residual-code restoration. ResCode forms compact expert clusters, coalesces residuals into cluster-shared low-rank bases, and learns compact codes to accurately recover the original routed outputs. ResCode provides the state-of-the-art performance for compressing SMoE models on various tasks, achieving up to 2.43 percentage points higher average accuracy than the strongest baseline.

How can question answering (QA) systems determine whether a query is ambiguous? Ambiguity detection is essential in open-domain QA, as misclassification leads to either answering under the wrong interpretation or triggering unnecessary clarification. However, existing methods conflate answer diversity with ambiguity, leading to inaccurate predictions. They also process all queries uniformly, resulting in wasteful computation. We propose ARCHIVE, an accurate and efficient framework that detects ambiguity via *logical conflict*: a query is ambiguous if its valid answers cannot all be true under a single interpretation. ARCHIVE combines a lightweight early-exit encoder for surface-detectable cases with a conflict modeling module that captures logical relations among answers, reinforced by an invariance objective for robustness to noisy answer sets. We also present QuireQA, a 4,703-query benchmark spanning factoid, non-factoid, and ill-formed queries. Experiments show that ARCHIVE outperforms competitors, improving F1-amb and F1-unamb by up to 10.4 and 21.6 points, respectively, while operating 16× faster than the best competitor.

Given a query, how can a QA system ask an informative clarifying question to accurately answer it? Open-domain conversational question answering requires not only detecting when a query is ambiguous but also asking clarifying questions that efficiently uncover the latent intent of the user. However, existing approaches either rely on prompt-only clarify-or-answer decisions with limited robustness or distill clarification trajectories without explicitly modeling intent convergence. In this paper, we propose D4CQ (Data-Efficient Diagnosticity based Dynamic Distillation for Clarifying Question), a data-efficient framework for learning diagnostic clarifying questions through dynamic distillation. Our approach distills a structured refinement protocol, prioritizes informative clarifications via diagnosticity-based reweighting, and improves sample efficiency through hindsight relabeling. Empirical results demonstrates that D4CQ consistently outperforms strong baselines, while maintaining comparable interaction budgets. Our code is available at https://github.com/ddddcq/d4cq.

How can we share parameters within large language models to significantly reduce memory costs while preserving accuracy? While parameter sharing is a promising solution to the memory overhead of large language models, existing methods rely on naive grouping and fail to correct sharing-induced discrepancies. We propose an accurate and efficient parameter sharing framework, SharVeT (Similarity-aware sharing with Vector-based Tuning), which performs similarity-based grouping to ensure accurate sharing, allocates parameters adaptively to preserve diversity within each group, and applies lightweight refinement with knowledge distillation to correct sharing-induced discrepancies. Experiments show that SharVeT outperforms existing sharing methods, achieving up to 32.1% lower perplexity and 23.3% higher few-shot reasoning accuracy.

How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization schemes that have strong expressiveness and optimizability, respectively. However, neither scheme leverages both advantages. In this paper, we propose UniQuan_F (Unified Quantization with Flexible Mapping), an accurate quantization method for LLMs. UniQuan_F harnesses both strong expressiveness and optimizability by unifying the flexible mapping technique in UQ and BCQ’s non-uniform quantization levels. We propose unified initialization, and local and periodic mapping techniques to optimize the parameters in UniQuan_F precisely. After optimization, our unification theorem removes computational and memory overhead, allowing us to utilize the superior accuracy of UniQuan_F without extra deployment costs induced by the unification. Experimental results demonstrate that UniQuan_F outperforms existing UQ and BCQ methods, achieving up to 4.60% higher accuracy on GSM8K benchmark.