Ruiyang Zhang

Author directory

Other people with similar names: Ruiyang Zhang, Ruiyang Zhang


2026

When a language-model agent fills tool-call arguments, the argument value is the action: 转一万二 ("transfer twelve thousand") must become transfer(12000), and 两斤苹果 ("two jin of apples") must become order(weight_kg=1.0). We show that today’s LLMs silently corrupt such values when Chinese locale semantics must be converted into canonical arguments — producing schema-valid, confidently wrong actions that no validator catches. We introduce ZhToolLocale, a diagnostic benchmark with machine-computed gold values (lunar-calendar libraries, deterministic numeral and unit parsers), paraphrase triplets, and a pre-registered audit protocol; we report 95% bootstrap confidence intervals and significance tests for every headline claim. Across six models from five families — including two Chinese-trained — we find: (1) lunar-calendar conversion degrades significantly in every model (all p ≤ .031) and catastrophically in five of six (0–28%, vs. ~100% on fixed-date controls); a pre-registered matched-pair control refuted our initial memorization hypothesis — explicit lunar dates are, if anything, easier than festival names — indicating genuine conversion difficulty rather than recall; (2) Chinese-centric training confers a sizable advantage at the large tier (deepseek-chat: 93.3% overall, 81% lunar — the strongest model tested) but does not rescue the 7B tier (qwen2.5: lunar 3%); (3) paraphrase disagreement detects scattered errors (precision up to 0.92) but is structurally blind to consistent mis-conversion, which we quantify; (4) a deterministic converter restores 94.8–100% accuracy with significant gains for three models (all p ≤ .0012), but routing is tier-governed: frontier models route voluntarily (93–99%), a mid-tier model improves only under an explicit mandate, and two 7–8B models cannot be made to route at all (gains non-significant) — implying that deployable-tier safety must be architectural rather than prompted. We release the benchmark as a generator whose golds are computed on demand, with a per-category deployment card.
Instruction-tuned language models refuse harmful requests far less reliably in lower-resource languages than in English. That behavioural gap is well documented; its representational basis is not. In English, refusal is known to be mediated by a single linear direction in the residual stream. We ask whether that direction is one object shared across languages or many language-specific ones. Using only released multilingual safety benchmarks, we extract per-language, per-layer difference-in-means refusal directions in 10 typologically diverse languages for two open instruction-tuned models, measure their pairwise alignment against a matched non-safety control direction, and test causally whether ablating or adding the English direction changes refusal behaviour in the other nine languages. We find no evidence for a shared, refusal-specific, causally transferable direction. In Qwen2.5-7B the per-language refusal directions are only weakly aligned (median cosine 0.38), just above a matched non-safety control (0.33; difference 0.05, CI [0.01, 0.09]), and the English direction—though ablating it removes 61.9 pp of English refusal—changes refusal in 0 of nine other languages. In Llama-3.2-3B the directions are strongly aligned (0.84), but so is the control (0.86; difference -0.02), so that alignment is a generic property of the model’s multilingual geometry, not something specific to safety—and the English direction still transfers to zero of nine languages. Geometric alignment thus does not imply a shared causal mechanism: refusal does not occupy a shared, transferable multilingual subspace in either model. We pre-register all thresholds and decision rules before touching activations, and release a verifier that re-derives every reported number from persisted activations and labels.
Search
Co-authors
    Venues
    Fix author