Haiyan Zhao
Other people with similar names: Haiyan Zhao
Unverified author pages with similar names: Haiyan Zhao
2026
Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steering
Haiyan Zhao | Xuansheng Wu | Fan Yang | Bo Shen | Ninghao Liu | Mengnan Du
Findings of the Association for Computational Linguistics: EACL 2026
Haiyan Zhao | Xuansheng Wu | Fan Yang | Bo Shen | Ninghao Liu | Mengnan Du
Findings of the Association for Computational Linguistics: EACL 2026
Linear concept vectors effectively steer LLMs, but existing methods suffer from noisy features in diverse datasets that undermine steering robustness. We propose Sparse Autoencoder-Denoised Concept Vectors (SDCV), which selectively keep the most discriminative SAE latents while reconstructing hidden representations. Our key insight is that concept-relevant signals can be explicitly separated from dataset noise by scaling up activations of top-k latents that best differentiate positive and negative samples. Applied to linear probing and difference-in-mean, SDCV consistently improves steering success rates by 4-16% across six challenging concepts, while maintaining topic relevance.