Yi Chen
Author directoryOther people with similar names: yi Chen, Yi Chen, Yi Chen, Yi Chen, Yi Chen
Unverified author pages with similar names: Yi Chen
2026
DarterSD: Dynamic Token-Weighted Distillation for High-Efficient Speculative Decoding
Zhaoning Zhang | Xu Baizhou | Libo Zhang | Yi Chen | Zhiliang Tian | Zhen Huang | Dongsheng Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhaoning Zhang | Xu Baizhou | Libo Zhang | Yi Chen | Zhiliang Tian | Zhen Huang | Dongsheng Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Speculative decoding accelerates autoregressive LLM generation with a lightweight draft model, yet existing distillation methods treat all historical tokens equally despite their divergent contributions to next-token prediction. We discover that attention in draft models is extremely concentrated: the top-3 historical tokens capture 80% of attention weight, while over 90% of tokens contribute negligibly. This uniform training paradigm forces the draft model to learn from noisy, low-importance tokens, diluting the learning signal from informative positions and limiting the alignment between draft and target distributions. To address this, we propose DarterSD, a dynamic token-weighted distillation framework for efficient speculative decoding. DarterSD employs a learnable sigmoid gate in self-attention to compute per-token importance scores, and designs a multi-objective distillation loss comprising distribution alignment, logit matching, and gate regularization to focus training on high-informative tokens. We further introduce a lightweight Boolean-mask denoising strategy at inference to suppress contextual noise, without modifying the target model’s verification pipeline. Experiments show DarterSD delivers a lossless speedup up to 5.52x , outperforming EAGLE3 (4.67x), and boosts the average accepted token length by 12.9%.