Ruiyang Zhang

Author directory

Other people with similar names: Ruiyang Zhang, Ruiyang Zhang

Unverified author pages with similar names: Ruiyang Zhang


2026

Machine translation of English web text is a default recipe for building multilingual and low-resource corpora. We show that this recipe systematically damages the structured spans embedded in web text—URLs, email addresses, numbers and identifiers, code, markup and quotation delimiters—and that fluency metrics do not reveal the damage. Translating 56,163 span-bearing segments from a CC-MAIN-2026-30 Common Crawl sample into nine typologically diverse languages with NLLB-200, we find that 12.6% of 73,881 structured spans do not survive, ranging from 91.1% for numbers to 50.6% for quotation delimiters. Damage is strongly language-dependent (18.7 points), but in the opposite direction to the obvious prediction: the highest-resource targets are damaged most, an inversion that survives controlling for the systems’ tendency to copy low-resource input through untranslated. Round-trip fluency barely tracks any of this (ρ = 0.48 across languages, p = 0.19; ρ = 0.25 per segment). We pre-registered the study, report the refuted hypothesis alongside the supported ones, and release the annotated parallel-span corpus with an independent re-derivation of every number.
When a language-model agent fills tool-call arguments, the argument value is the action: 转一万二 ("transfer twelve thousand") must become transfer(12000), and 两斤苹果 ("two jin of apples") must become order(weight_kg=1.0). We show that today’s LLMs silently corrupt such values when Chinese locale semantics must be converted into canonical arguments — producing schema-valid, confidently wrong actions that no validator catches. We introduce ZhToolLocale, a diagnostic benchmark with machine-computed gold values (lunar-calendar libraries, deterministic numeral and unit parsers), paraphrase triplets, and a pre-registered audit protocol; we report 95% bootstrap confidence intervals and significance tests for every headline claim. Across six models from five families — including two Chinese-trained — we find: (1) lunar-calendar conversion degrades significantly in every model (all p ≤ .031) and catastrophically in five of six (0–28%, vs. ~100% on fixed-date controls); a pre-registered matched-pair control refuted our initial memorization hypothesis — explicit lunar dates are, if anything, easier than festival names — indicating genuine conversion difficulty rather than recall; (2) Chinese-centric training confers a sizable advantage at the large tier (deepseek-chat: 93.3% overall, 81% lunar — the strongest model tested) but does not rescue the 7B tier (qwen2.5: lunar 3%); (3) paraphrase disagreement detects scattered errors (precision up to 0.92) but is structurally blind to consistent mis-conversion, which we quantify; (4) a deterministic converter restores 94.8–100% accuracy with significant gains for three models (all p ≤ .0012), but routing is tier-governed: frontier models route voluntarily (93–99%), a mid-tier model improves only under an explicit mandate, and two 7–8B models cannot be made to route at all (gains non-significant) — implying that deployable-tier safety must be architectural rather than prompted. We release the benchmark as a generator whose golds are computed on demand, with a per-category deployment card.
Instruction-tuned language models refuse harmful requests far less reliably in lower-resource languages than in English. That behavioural gap is well documented; its representational basis is not. In English, refusal is known to be mediated by a single linear direction in the residual stream. We ask whether that direction is one object shared across languages or many language-specific ones. Using only released multilingual safety benchmarks, we extract per-language, per-layer difference-in-means refusal directions in 10 typologically diverse languages for two open instruction-tuned models, measure their pairwise alignment against a matched non-safety control direction, and test causally whether ablating or adding the English direction changes refusal behaviour in the other nine languages. We find no evidence for a shared, refusal-specific, causally transferable direction. In Qwen2.5-7B the per-language refusal directions are only weakly aligned (median cosine 0.38), just above a matched non-safety control (0.33; difference 0.05, CI [0.01, 0.09]), and the English direction—though ablating it removes 61.9 pp of English refusal—changes refusal in 0 of nine other languages. In Llama-3.2-3B the directions are strongly aligned (0.84), but so is the control (0.86; difference -0.02), so that alignment is a generic property of the model’s multilingual geometry, not something specific to safety—and the English direction still transfers to zero of nine languages. Geometric alignment thus does not imply a shared causal mechanism: refusal does not occupy a shared, transferable multilingual subspace in either model. We pre-register all thresholds and decision rules before touching activations, and release a verifier that re-derives every reported number from persisted activations and labels.
Document-AI pipelines increasingly begin by converting a PDF to text with a parser—classical extractors, OCR, layout models, or vision-language models—before any retrieval or reasoning. Overall-quality metrics (edit distance, TEDS) score these parsers as if all characters mattered equally. They do not: a corrupted number, date, identifier, or table cell changes meaning while barely moving a similarity score. We isolate this failure with a span-typed diagnostic over six parsers, ten structured span types, and two conditions (digital and scanned), on a controlled corpus with exact ground truth plus a real arXiv subset. Three findings. (i) Preservation is highly parser- and type-dependent: the best parser preserves 99.4% of spans while others fail specific types catastrophically (Docling drops markup to 6.0%; MarkItDown returns nothing on scanned pages), so “which parser” is a per-span-type risk decision, not a single score. (ii) Document-level text similarity does not predict span loss (Spearman ρ ≤ 0.46 across all parsers and conditions; ≈ 0 for VLMs), so standard metrics are blind to it. (iii) Downstream, corrupted spans are dangerous precisely when the parse looks successful: among questions whose evidence span was corrupted but whose document still returned text, 96.5% are answered with a confident wrong value rather than an abstention—an undetectable error—whereas total parse failure is comparatively safe because it forces an abstention. We release the resource behind these numbers: 566,904 per-span verdicts over 660 documents and 7,560 parser outputs, under CC-BY, with an independent verifier (49,720 checks) that re-derives every reported number.