Ruiyang Zhang

Author directory

Papers on this page may belong to the following people: Ruiyang Zhang, Ruiyang Zhang, Ruiyang Zhang


2026

It is widely assumed that translating an attack into a lower-resource language makes a language model more vulnerable, and it is natural to expect the same for tool-using agents: an English-built runtime monitor should lose coverage when the same indirect prompt injection is mounted in another language. We test this directly. Using AgentDojo with two models, six languages (English, French, Chinese, Arabic, Hindi, Swahili), and a fixed, strictly paired 320-scenario sample per cell (3,840 attack trajectories, zero execution errors), we measure attack success, attack-pattern entropy, and cross-lingual transfer of a mined tool-call monitor. Every reported number is re-derived from raw trajectories by an independent verifier (965/965 checks pass). The expected result does not appear. Translating the injection lowers attack success by 3–6× while task utility rises; attack tool-call structure does not disperse with linguistic distance; and an Englishmined monitor loses no recall across languages once it is evaluated out-of-sample. We further show that the natural in-sample evaluation of a mined monitor manufactures a clean-looking coverage-bound failure that is entirely an artifact of memorization, and that three machinetranslation pipeline defects each independently manufacture a spurious “safety does not travel” effect. A pre-registered paraphrase control narrows but does not overturn the finding: a measurable, model-dependent minority (17–33%) of the drop is surface-form instruction fragility rather than language. We conclude that the multilingual-jailbreak result does not reproduce at the agent layer on this benchmark, and we distill the measurement pitfalls that would have produced the opposite, and wrong, conclusion.
Document-AI pipelines increasingly begin by converting a PDF to text with a parser—classical extractors, OCR, layout models, or vision-language models—before any retrieval or reasoning. Overall-quality metrics (edit distance, TEDS) score these parsers as if all characters mattered equally. They do not: a corrupted number, date, identifier, or table cell changes meaning while barely moving a similarity score. We isolate this failure with a span-typed diagnostic over six parsers, ten structured span types, and two conditions (digital and scanned), on a controlled corpus with exact ground truth plus a real arXiv subset. Three findings. (i) Preservation is highly parser- and type-dependent: the best parser preserves 99.4% of spans while others fail specific types catastrophically (Docling drops markup to 6.0%; MarkItDown returns nothing on scanned pages), so "which parser" is a per-span-type risk decision, not a single score. (ii) Document-level text similarity does not predict span loss (Spearman ρ ≤ 0.46 across all parsers and conditions; ≈ 0 for VLMs), so standard metrics are blind to it. (iii) Downstream, corrupted spans are dangerous precisely when the parse looks successful: among questions whose evidence span was corrupted but whose document still returned text, 96.5% are answered with a confident wrong value rather than an abstention—an undetectable error—whereas total parse failure is comparatively safe because it forces an abstention. We release the resource behind these numbers: 566,904 per-span verdicts over 660 documents and 7,560 parser outputs, under CC-BY, with an independent verifier (49,720 checks) that re-derives every reported number.