Yusuke Sakai

Author directory

Other people with similar names: Yusuke Sakai

Unverified author pages with similar names: Yusuke Sakai


2026

Contrastive decoding methods such as DoLa improve the factuality of Large Language Models (LLMs) by contrasting the output distributions of mature and premature layers. However, DoLa’s dynamic layer selection relies solely on divergences in output vocabulary distributions. In this work, we propose three attention-guided strategies—Attention-JSD, Attention-Entropy-Max, and Attention-Entropy-Min—that leverage structural information carried by internal self-attention mechanisms as a signal for layer selection. Experimental results on TruthfulQA demonstrate that our strategies, particularly Attention-JSD and Attention-Entropy-Min, consistently outperform the original DoLa. We observe significant gains on multi-answer metrics (MC2 and MC3), suggesting that attention distributions can provide a more sensitive signal for resolving factual knowledge than output vocabulary distributions.