Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel. However, the verification stage often becomes the dominant computational bottleneck, especially for long-context inputs and mixture-of-experts (MoE) models. Existing sparsification methods are designed primarily for token-by-token autoregressive decoding to remove substantial computational redundancy in LLMs. This work systematically adopts different sparse methods on the verification stage of the speculative decoding and identifies structured redundancy across multiple dimensions. Based on these observations, we propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost. The framework further incorporates an inter-draft token and inter-layer retrieval reuse strategy to further reduce redundant computation without introducing additional training. Extensive experiments across summarization, question answering, and mathematical reasoning datasets demonstrate that the proposed methods achieve favorable efficiency-accuracy trade-offs and can be translated into practical latency reduction with optimized sparse kernels.

Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architectural modifications or optimizer adjustments. However, these approaches lack systematic optimization of weight patterns during training. Weight pattern refers to the distribution and relative magnitudes of weight parameters in a neural network. To address this issue, we propose a Weight Scaling method called WISCA to enhance training efficiency and model quality by strategically improving neural network weight patterns—without changing network structures. By rescaling weights while preserving model outputs, WISCA indirectly optimizes the model’s training trajectory. Experiments demonstrate that WISCA significantly improves convergence quality (measured by generalization capability and loss reduction), particularly in LLMs with Grouped Query Attention (GQA) architectures and LoRA fine-tuning tasks. Empirical results show 5.6% average improvement on zero-shot validation tasks and 2.12% average reduction in training perplexity across multiple architectures.

Recent work reports large gains for LLM agents on low-budget molecular optimization, with 2-3 times AUC improvements over reinforcement-learning and genetic-algorithm baselines on the PMO benchmark and similar gains on protein-targeted docking benchmarks. We investigate whether these gains reflect genuine optimization or benchmark artifacts, by re-evaluating a representative LLM agent under controlled blinding protocols and against a trivial random-walk baseline. We find that on PMO at the 50-call budget, the agent’s apparent advantage collapses below classical baselines once task names and curated reference compounds are removed from its prompt; a 2 times 2 ablation shows that retrieval from a small reference dictionary, not iterative optimization, is the dominant driver of the headline number. On a new protein-targeted lead-optimization benchmark with an AutoDock Vina composite oracle, a single-edit halogen/methyl random-walk mutator outperforms the LLM agent, a graph-based GA, and a non-iterative GPT baseline on every aggregate metric—a property of the oracle landscape under tight budgets, not of LLM agents per se. We release both protocols (name-blinded PMO scoring; random-walk lower bound for Vina-based protein benchmarks) as standard sanity checks for future LLM-agent papers in molecular optimization.