Weiqing Zeng

Author directory

2026

Prompt rewrites can prescribe a change in an answer, not merely a change in wording. We introduce ORBIT-SIT, a framework that assigns each rewrite an explicit action on a task’s output space and separately measures correctness against transformed gold answers and consistency with transformed model predictions. Across eight operators, five datasets, and three instruction-tuned models, high task accuracy does not ensure compliance: Phi-4-mini achieves 90.7% accuracy on SST-2 but only 16.4% under a label-swap instruction. Deterministic validation and a three-annotator audit check the operator contracts. Non-deceptive binary controls retain a 17.8-percentage-point gap, and separating inference from transformation improves compliance, implicating joint execution rather than refusal alone. OrbitAlign, a quantized low-rank adaptation objective, reduces gold-grounded risk by 25.4 points on a 400-item BoolQ repair pool. The recipe lowers risk in 39 of 40 cells on two additional backbones, and same-action transfer across binary tasks reduces risk by 12.9–15.1 points. The binary tasks provide the primary evidence; structured tasks remain extensibility diagnostics. ORBIT-SIT makes declared prompt actions measurable and partly repairable without equating compliance with general understanding.
Current RAG evaluation measures answer quality under fixed evidence but cannot detect systems that ignore evidence changes or fall back to parametric knowledge. We introduce Dynamic Evidence Contracts, a behavioral evaluation paradigm that specifies expected RAG behavior under four evidence interventions (deletion, conflict, distraction, permutation), and the Intervention Compliance Score (ICS) to quantify compliance. Evaluation on five datasets reveals that all evaluated open-source RAG pipelines achieve moderate conflict detection (CDR=0.283–0.847) via passive NLI-detected refutation signals, but lack explicit conflict acknowledgment. We further present DOVE-RAG, a minimal reference instantiation showing that explicit conflict acknowledgment yields higher CDR (0.505–0.999), up to 1.6× ICS improvement, and 2.0× citation recall gains. Ablation reveals that verification without structured output reorganization yields negligible improvement, suggesting that evidence compliance requires an integrated pipeline.
When large language models serve as evaluation infrastructure, progress cannot be reduced to a single accuracy number. We study winning-argument (WA) detection as a stress test: can one system jointly improve accuracy, enforce speaker-order invariance, emit rubric-conditioned diagnostics, improve a proper scoring rule, and cut inference cost? TAPIR freezes a survey-grounded rubric into a dual-artifact protocol, distils privileged signals by mode-split self-distillation, and predicts through an anti-symmetric scalar head paired with a structured ledger. An algebraic swap identity (Winning-Position Invariance, WPI=1) holds on every test pair, and a tied-order wrapper preserves it on the deployed router. On the standard 807-pair WA split, TAPIR beats our same-backbone Qwen3-14B re-implementation of Labruna et al.’s MS-PS-MLP, 65.80% vs. 58.12% (paired McNemar p=3.41×10−4), at a lower Brier score and 39.41× faster inference per pair. Reusing the frozen protocol without retuning carries the mechanism to a second task and a second backbone, both at WPI=1. Under domain shift it only matches length-only controls, so we report contract preservation rather than superiority. The contribution is the joint contract under a paired protocol, with an explicit amortisation boundary, not a leaderboard delta.
Large language models (LLMs) are increasingly used to support scientific peer review, yet most review systems treat the manuscript as a trustworthy input channel. This assumption fails when manuscripts contain hidden instructions, rubric-replacing text, citation pollution, fabricated evidence, or formatting attacks aimed at generative reviewers. We formalize Untrusted-Manuscript Peer Review, where scientific judgment should remain invariant to non-scientific perturbations and every certified critique must be bound to trust-screened evidence. We introduce SEAL-PR, a five-stage architecture combining a forensic span parser, a five-label trust lattice, a typed evidence-audit graph, sealed/shadow dual-view reviewing, and evidence-constrained decoding. We separate two classes of claim throughout. Structurally, no manuscript span can enter the trusted-instruction channel and no certified critique can cite an inadmissible span, so evidence admissibility is 1.000 by construction rather than by measurement. Empirically, on SEAL-PR-BENCH—a paired benchmark of 5129 papers across 7 venues, 129 independent evaluation manuscripts, 1290 clean+attacked records and 9 attack families—SEAL-PR attains 0.994 clean expert weakness recall, preserves 0.993 under attack, and cuts the text-acceptance attack rate from 0.866 to 0.000, without reviewer-LLM fine-tuning. Because the corpus predates our backbones, absolute EWR is an in-corpus ceiling; the paired clean–attacked contrast is the quantity we treat as informative. SEAL-PR reframes AI peer review from fluent review generation to locally isolated, evidence-audited scientific assistance.