In-Distribution Steering: Balancing Control and Coherence in Language Model Generation

Arthur Vogels, Benjamin Wong, Yann Choho, Annabelle Blangero, Milan Bhan


Abstract
Activation steering methods control large language model (LLM) behavior by modifying internal activations at inference time. However, most existing activation steering methods rely on a fixed steering strength, leading to either insufficient control or unadapted intervention that degrades text plausibility and coherence. We introduce In-Distribution Steering (IDS), a novel method that adapts steering strength based on the input data distribution in representation space. IDS dynamically adjusts interventions according to how far a given input lies within the distribution, enabling adaptive intervention and generation stability during text generation. Experiments demonstrate that IDS achieves strong accuracy on classification tasks while producing coherent text without collapse, making IDS particularly well suited for real-world applications.
Anthology ID:
2026.lrec-1.163
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
2076–2089
Language:
External URL:
https://lrec.elra.info/lrec2026-main-163
DOI:
10.63317/4629fxavjicu
Bibkey:
Cite (ACL):
Arthur Vogels, Benjamin Wong, Yann Choho, Annabelle Blangero, and Milan Bhan. 2026. In-Distribution Steering: Balancing Control and Coherence in Language Model Generation. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 2076–2089, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
In-Distribution Steering: Balancing Control and Coherence in Language Model Generation (Vogels et al., LREC 2026)
Copy Citation: