Melissa Li

Author directory

2026

Stateful agents increasingly leverage execution logs, tool outputs, feedback, and summaries from prior interactions to inform future behavior. Despite this, safety evaluations typically evaluate agents using either a singular fixed prior history or an empty baseline, via a uniform history rendering. We argue that such evaluations only illustrate a conditional snapshot of model behavior and fail to demonstrate whether these conclusions generalize across plausible prior histories and history renderings. We use the Inspect Evals agentic misalignment benchmark, finding that among eight benign prior histories, a single Opus 4.1 history significantly and repeatedly increased harmful-action rates, whereas the remaining seven alternatives exhibited no such increase relative to the empty history condition. While additional rollout replications improved statistical confidence, they did not improve the representativeness of the overall sample. Conversely, all eight Qwen3-30B histories resulted in an average decrease of 26.3 percentage points in observed harmful actions. Furthermore, we examined four distinct history renderings of 15 benign prior histories across 900 Opus blackmail rollouts. History renderings with tool results and neutral summaries reduced mean harmful-action rates by 10.0 and 15.0 percentage points, respectively. These results emphasize the necessity for evaluations to incorporate diverse, plausible prior histories and history renderings to establish that their findings generalize.