Does Safety Reasoning Travel? A Multilingual Measurement of Prompt-Injection Attacks and Runtime Monitors in Tool-Using LLM Agents

Ruiyang Zhang


Abstract
It is widely assumed that translating an attack into a lower-resource language makes a language model more vulnerable, and it is natural to expect the same for tool-using agents: an English-built runtime monitor should lose coverage when the same indirect prompt injection is mounted in another language. We test this directly. Using AgentDojo with two models, six languages (English, French, Chinese, Arabic, Hindi, Swahili), and a fixed, strictly paired 320-scenario sample per cell (3,840 attack trajectories, zero execution errors), we measure attack success, attack-pattern entropy, and cross-lingual transfer of a mined tool-call monitor. Every reported number is re-derived from raw trajectories by an independent verifier (965/965 checks pass). The expected result does not appear. Translating the injection lowers attack success by 3–6× while task utility rises; attack tool-call structure does not disperse with linguistic distance; and an Englishmined monitor loses no recall across languages once it is evaluated out-of-sample. We further show that the natural in-sample evaluation of a mined monitor manufactures a clean-looking coverage-bound failure that is entirely an artifact of memorization, and that three machinetranslation pipeline defects each independently manufacture a spurious “safety does not travel” effect. A pre-registered paraphrase control narrows but does not overturn the finding: a measurable, model-dependent minority (17–33%) of the drop is surface-form instruction fragility rather than language. We conclude that the multilingual-jailbreak result does not reproduce at the agent layer on this benchmark, and we distill the measurement pitfalls that would have produced the opposite, and wrong, conclusion.
Anthology ID:
2026.oracle-main.13
Volume:
The Proceedings of the Workshop on Open Reasoning Across Cultures Languages (ORACLE)
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Leonardo Ranaldi, Giulia Pucci, Federico Ranaldi, Marco Valentino, Alexandra Birch, Fabio Massimo Zanzotto
Venues:
ORACLE | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
194–204
Language:
URL:
https://aclanthology.org/2026.oracle-main.13/
DOI:
Bibkey:
Cite (ACL):
Ruiyang Zhang. 2026. Does Safety Reasoning Travel? A Multilingual Measurement of Prompt-Injection Attacks and Runtime Monitors in Tool-Using LLM Agents. In The Proceedings of the Workshop on Open Reasoning Across Cultures Languages (ORACLE), pages 194–204, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Does Safety Reasoning Travel? A Multilingual Measurement of Prompt-Injection Attacks and Runtime Monitors in Tool-Using LLM Agents (Zhang, ORACLE 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.oracle-main.13.pdf