Controlling Language and Style of Multi-lingual Generative Language Models with Control Vectors

Julius Leino, Jussi Karlgren


Abstract
Control vectors have recently gained popularity as a method for steering transformer-based generative language models. This paper contributes to this path of research by evaluating the robustness of these control vectors in multi- and cross-lingual question-answering settings mimicking the real-world deployment scenario, where models are expected to generate answers to challenging questions. We present a set of experiments to demonstrate that a control vector approach can be used to shift the output of a generative language model from one language to another, and to exercise stylistic control of the output across languages. Overall, we find that the control vector approach offers a relatively lightweight and effective path for developing methods to control the output of multilingual language models with multiple design choices affecting the real-world control performance.
Anthology ID:
2025.nejlt-1.1
Volume:
Northern European Journal of Language Technology, Volume 11
Month:
December
Year:
2025
Address:
Linköping, Sweden
Editor:
Marcel Bollmann
Venue:
NEJLT
SIG:
Publisher:
Linköping University Electronic Press
Note:
Pages:
1–26
Language:
URL:
https://aclanthology.org/2025.nejlt-1.1/
DOI:
10.3384/nejlt.2000-1533.2025.5888
Bibkey:
Cite (ACL):
Julius Leino and Jussi Karlgren. 2025. Controlling Language and Style of Multi-lingual Generative Language Models with Control Vectors. Northern European Journal of Language Technology, 11:1–26.
Cite (Informal):
Controlling Language and Style of Multi-lingual Generative Language Models with Control Vectors (Leino & Karlgren, NEJLT 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.nejlt-1.1.pdf