Evaluating the Semantic Specificity of Representation Steering in Language Models

Moy Yuan, Andreas Vlachos


Abstract
Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by superficial label overrides, creating a false impression of reasoning circuit repairs. In this work, we propose Cross-Rule Transfer (CRT), a diagnostic framework that audits representational interventions by evaluating them on rule families where the model is natively competent. Evaluating late-layer LRS for a widespread logical failure, contradiction blindness, reveals that the intervention merely injects a global label bias: applying the steering vector to rules the model already handles correctly (99.6% baseline) degrades performance to 40.4% by forcing false contradiction predictions. We support this diagnosis with four complementary controls (direct logit bias equivalence, control vector label-flipping, cross-model grafting, and early-layer steering checks), providing a rigorous methodology to distinguish genuine reasoning repairs from superficial label overrides.
Anthology ID:
2026.findings-emnlp.562
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
11853–11871
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.562/
DOI:
Bibkey:
Cite (ACL):
Moy Yuan and Andreas Vlachos. 2026. Evaluating the Semantic Specificity of Representation Steering in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 11853–11871, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Evaluating the Semantic Specificity of Representation Steering in Language Models (Yuan & Vlachos, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.562.pdf