Yong Wang
Author directoryOther people with similar names: Yong Wang, Yong Wang, Yong Wang
Unverified author pages with similar names: Yong Wang
2026
Debiasing as a Measurement Intervention: Calibrated Ties and Resolution Loss in LLM-as-a-Judge Evaluation
Liang Zhao | Yong Wang | Jiangzhe Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Liang Zhao | Yong Wang | Jiangzhe Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
LLM-as-a-judge protocols are commonly debiased by instructing judges to ignore presentation cues such as citation formatting, source labels, and evidence-display style. We show that this intervention can suppress bias while damaging the resolution of the measurement instrument. We introduce TraceJudgeBench, a diagnostic benchmark for auditing citation-like artifacts in RAG and agent-workflow evaluation, covering content-equivalent pairs, citation ablations, correctness conflicts, human-validated soft and moderate quality gaps, prompt-strength ladders, decoupled judging, and a controlled workflow-ranking probe. Across GPT-5.5, Claude Sonnet 4.6, and DeepSeek V4-Flash, stronger anti-citation prompts reduce worse-cited wins from up to 50.5% to 0%; yet some operating points already convert validated moderate-gap decisions into Tie before the strict stress-test endpoint, while correctness-conflict accuracy remains at or above 93.0%. Open-weight Qwen2.5-14B-Instruct-AWQ and Gemma-3-12B-IT runs reproduce the central moderate-gap frontier, with TRACE-style decoupling recovering 96.5–98.5% better-plain resolution. Human validation separates three meanings of Tie: correct equivalence Tie, calibrated soft-boundary Tie, and resolution-destroying Tie on validated quality gaps. We frame debiasing as a measurement intervention whose bias suppression, resolution retention, Tie cost, and protocol cost must be reported jointly. We include TraceJudgeBench as a supplementary artifact with anonymized splits, prompts, raw judge outputs, validation summaries, and analysis scripts.