Yihang Chen
Author directoryOther people with similar names: Yihang Chen
2026
Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification
Yihang Chen | Pin Qian | Su Wang | Chong Peng | Huan Xu | Shuaiting Li | Yiqi Sun
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Yihang Chen | Pin Qian | Su Wang | Chong Peng | Huan Xu | Shuaiting Li | Yiqi Sun
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Personalized agents must decide whether retrieved user memory should be used, ignored, updated, or queried before it affects a task. We develop an empirical audit protocol for structured intermediate outputs: auditing dataset shortcuts, isolating bundled prompt changes, testing answer-associated labels and decomposed evidence, and retaining provider-level execution failures. A 480-example synthetic development set initially suggested large gains from a state-structured prompt bundle, but TF-IDF diagnostics showed lexical separability and no positive standalone Ignore cases. We therefore build a frozen 160-example controlled counterfactual set with 40 matched four-way families and rule-derived reference policies. There, exposing four state definitions improves accuracy, but adding an isolated explicit state-output field does not significantly improve policy accuracy for Llama-3.3-70B and gives only a marginal, non-significant gain for GPT-OSS-120B. Supplying benchmark-associated state labels shifts policy predictions; because those labels deterministically map to policies, this is label-conditioning, not evidence of a faithful mechanism. Family-level and seed-stability analyses show that example accuracy overstates counterfactual consistency. A decomposed semantic-evidence follow-up does not improve routing for the cleanly evaluated endpoint, while the GPT-OSS condition fails provider-side request validation. We evaluate policy classification only.
Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG
Pin Qian | Su Wang | Yihang Chen | Wenxuan Xu | Xiaoyuan Wang | Qiaolin Yu | Shuhuai Lin | Sipeng Zhang | Junxian You | Xinpeng Wei
Findings of the Association for Computational Linguistics: EMNLP 2026
Pin Qian | Su Wang | Yihang Chen | Wenxuan Xu | Xiaoyuan Wang | Qiaolin Yu | Shuhuai Lin | Sipeng Zhang | Junxian You | Xinpeng Wei
Findings of the Association for Computational Linguistics: EMNLP 2026
Cited RAG evaluation often treats visible sources as a grounding signal, but a real, topically relevant citation can still under-warrant the attached wording. We study this diagnostic failure as citation laundering: a related source is presented as warrant for an over-strong claim. We introduce ForceBench, a contrastive stress test for evidence-force calibration. Each item holds a cited passage fixed and pairs an evidence-calibrated claim with a localized force-raised variant across five operational axes: relation, modality, scope, temporal validity, and numeric specificity. A calibrated evaluator should score the evidence-calibrated claim higher. Headline experiments use a fixed, locality-filtered 198-pair evaluation set. A citation-presence sanity check is uninformative by design; token and entity overlap still violate monotonicity on 32.8–36.4% of pairs. Across four reported model judges, standard generic support prompting is insufficient for this force-calibration stress test (aggregate MVR 47.2%), while explicit warrant-strength prompting lowers MVR to 24.5% but remains imperfect. We release the benchmark, prompts, outputs, and plug-in pipeline so citation evaluators can report monotonicity violation rate and force sensitivity alongside conventional support metrics.