Ruiyang Zhang
Author directoryOther people with similar names: Ruiyang Zhang, Ruiyang Zhang
2026
Silent Locale Corruption: Diagnosing and Mitigating Chinese-Locale Semantic Failures in LLM Tool Calling
Ruiyang Zhang
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Ruiyang Zhang
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
When a language-model agent fills tool-call arguments, the argument value is the action: 转一万二 ("transfer twelve thousand") must become transfer(12000), and 两斤苹果 ("two jin of apples") must become order(weight_kg=1.0). We show that today’s LLMs silently corrupt such values when Chinese locale semantics must be converted into canonical arguments — producing schema-valid, confidently wrong actions that no validator catches. We introduce ZhToolLocale, a diagnostic benchmark with machine-computed gold values (lunar-calendar libraries, deterministic numeral and unit parsers), paraphrase triplets, and a pre-registered audit protocol; we report 95% bootstrap confidence intervals and significance tests for every headline claim. Across six models from five families — including two Chinese-trained — we find: (1) lunar-calendar conversion degrades significantly in every model (all p ≤ .031) and catastrophically in five of six (0–28%, vs. ~100% on fixed-date controls); a pre-registered matched-pair control refuted our initial memorization hypothesis — explicit lunar dates are, if anything, easier than festival names — indicating genuine conversion difficulty rather than recall; (2) Chinese-centric training confers a sizable advantage at the large tier (deepseek-chat: 93.3% overall, 81% lunar — the strongest model tested) but does not rescue the 7B tier (qwen2.5: lunar 3%); (3) paraphrase disagreement detects scattered errors (precision up to 0.92) but is structurally blind to consistent mis-conversion, which we quantify; (4) a deterministic converter restores 94.8–100% accuracy with significant gains for three models (all p ≤ .0012), but routing is tier-governed: frontier models route voluntarily (93–99%), a mid-tier model improves only under an explicit mandate, and two 7–8B models cannot be made to route at all (gains non-significant) — implying that deployable-tier safety must be architectural rather than prompted. We release the benchmark as a generator whose golds are computed on demand, with a per-category deployment card.
Does Safety Live in a Shared Multilingual Subspace? Cross-Lingual (Non-)Transfer of the Refusal Direction
Ruiyang Zhang
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Ruiyang Zhang
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Instruction-tuned language models refuse harmful requests far less reliably in lower-resource languages than in English. That behavioural gap is well documented; its representational basis is not. In English, refusal is known to be mediated by a single linear direction in the residual stream. We ask whether that direction is one object shared across languages or many language-specific ones. Using only released multilingual safety benchmarks, we extract per-language, per-layer difference-in-means refusal directions in 10 typologically diverse languages for two open instruction-tuned models, measure their pairwise alignment against a matched non-safety control direction, and test causally whether ablating or adding the English direction changes refusal behaviour in the other nine languages. We find no evidence for a shared, refusal-specific, causally transferable direction. In Qwen2.5-7B the per-language refusal directions are only weakly aligned (median cosine 0.38), just above a matched non-safety control (0.33; difference 0.05, CI [0.01, 0.09]), and the English direction—though ablating it removes 61.9 pp of English refusal—changes refusal in 0 of nine other languages. In Llama-3.2-3B the directions are strongly aligned (0.84), but so is the control (0.86; difference -0.02), so that alignment is a generic property of the model’s multilingual geometry, not something specific to safety—and the English direction still transfers to zero of nine languages. Geometric alignment thus does not imply a shared causal mechanism: refusal does not occupy a shared, transferable multilingual subspace in either model. We pre-register all thresholds and decision rules before touching activations, and release a verifier that re-derives every reported number from persisted activations and labels.