Shuaiting Li

Author directory

2026

Personalized agents must decide whether retrieved user memory should be used, ignored, updated, or queried before it affects a task. We develop an empirical audit protocol for structured intermediate outputs: auditing dataset shortcuts, isolating bundled prompt changes, testing answer-associated labels and decomposed evidence, and retaining provider-level execution failures. A 480-example synthetic development set initially suggested large gains from a state-structured prompt bundle, but TF-IDF diagnostics showed lexical separability and no positive standalone Ignore cases. We therefore build a frozen 160-example controlled counterfactual set with 40 matched four-way families and rule-derived reference policies. There, exposing four state definitions improves accuracy, but adding an isolated explicit state-output field does not significantly improve policy accuracy for Llama-3.3-70B and gives only a marginal, non-significant gain for GPT-OSS-120B. Supplying benchmark-associated state labels shifts policy predictions; because those labels deterministically map to policies, this is label-conditioning, not evidence of a faithful mechanism. Family-level and seed-stability analyses show that example accuracy overstates counterfactual consistency. A decomposed semantic-evidence follow-up does not improve routing for the cleanly evaluated endpoint, while the GPT-OSS condition fails provider-side request validation. We evaluate policy classification only.