2026

Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization. Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs. To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations. LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones. This decoupled design reduces optimization complexity without sacrificing outlier smoothing capability. Extensive experiments across the LLaMA-2/3, Mistral, Qwen3, and Mixtral families demonstrate that LIRQuant achieves state-of-the-art accuracy under the challenging W4A4KV4 setting. Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to 2.3× prefill speedup and 3.5× memory saving over FP16.
Enabling Large Language Models (LLMs) to perform tool-free Table Question Answering (TableQA) presents two critical challenges: "question-first" data synthesis often introduces logical hallucinations, while standard reinforcement learning (RL) suffers from sparse rewards over long reasoning chains. We propose ProbeTQA, a unified framework that couples execution-verified data construction with process-supervised RL. To mitigate synthesis hallucinations, ProbeTQA adopts an answer-first executable probing paradigm, in which an agent performs multi-round exploratory programming over real tables so that every synthesized instance is grounded in a verifiable witness program. To address reward sparsity, we convert verified code trajectories into natural-language reasoning traces and derive execution-grounded process rubrics directly from runtime states, providing reliable dense supervision. Using these rubrics, we optimize the model with Group Relative Policy Optimization (GRPO) under a Two-Stage Reward Curriculum, thereby internalizing complex tool-use reasoning into the model parameters. Experiments show that our 8B model achieves state-of-the-art tool-free performance on challenging benchmarks, attaining 65.8% accuracy on TableBench and outperforming GPT-5.4 and DeepSeek-v3.2 by 13.4 and 6.8 percentage points, respectively.
Large language models achieve remarkable performance via extended chain-of-thought (CoT) reasoning, yet this lengthy process incurs substantial inference overhead. Existing CoT compression methods struggle with inflexible manual length budgets, computationally expensive multi-stage training pipelines, and fragile scalability restricted to small models. We propose **HMPO** (**H**ybrid **M**edian-length **P**olicy **O**ptimization), a cost-effective, single-stage reinforcement learning framework. HMPO efficiently compresses CoT via three synergistic components: an *adaptive median-based budget* derived from successful rollouts to eliminate manual tuning, a *cosine-decay token reward* for smooth length penalization, and a *multiplicative reward formulation* that substantially mitigates trivial reward hacking by strictly prioritizing answer correctness. Trained exclusively on mathematical data, HMPO generalizes seamlessly across math, code, science, and instruction-following tasks. Extensive experiments scaling from 9B to 122B parameters across dense and Mixture-of-Experts (MoE) architectures demonstrate that HMPO achieves **19%–46% token compression with negligible accuracy degradation**, all while drastically reducing training costs compared to existing multi-stage baselines.
Safety alignment in large language models is markedly uneven across languages: behaviors learned from English preference data fail to transfer reliably into lower-resource languages, and existing remedies based on per-language preference data or pairwise translation pivots do not scale. We propose Hyperspherical Equilibrium for Aligned Languages (HEAL), a lightweight auxiliary objective that augments any DPO-style alignment pipeline without altering its supervised pairs. HEAL applies two opposing forces to the multilingual representations of the same prompt: a shrink term draws each non-English representation toward a stop-gradient English anchor, while a dilation term repels same-prompt language representations through a pairwise hinge. At equilibrium, representations occupy an English-centered shell-like region that supports cross-lingual transfer while preserving per-language distinctness. Across multiple backbones, HEAL substantially strengthens multilingual safety with the largest gains on the weakest languages, while preserving general capability and showing favorable out-of-distribution behavior on jailbreak evaluation.