LLM agents have made rapid progress on software engineering and ML research tasks, but these advances often assume access to a working runnable environment. For research artifacts released alongside published papers, setting up such an environment from a fresh machine remains a major bottleneck. Existing environment setup benchmarks do not cover the full scope of research-artifact deployment, which involves multi-language toolchains, system-level dependencies beyond containers (e.g., GPU/CUDA and kernel configurations), and legacy artifact compatibility. We introduce DeployBench, a multi-domain benchmark of 51 research-artifact deployment tasks spanning AI/ML, computer systems, and scientific computing, covering all these dimensions. Each task is verified by a hidden pipeline that executes the paper’s designated experiment and checks its outputs. Evaluating five state-of-the-art LLMs with OpenHands yields pass-rates from 7.8%-51.0%. Failures are dominated by a completion-judgment problem: 102 of 181 are agent-terminated self-stops, where the agent’s pre-finish checks validate a different or weaker target than the paper-specific task requires. DeployBench highlights the gap between current agents and autonomous deployment, and offers a realistic testbed for scientific research agents.

LLMs often fail to handle temporal knowledge conflicts–contradictions that arise when facts evolve over time within their training data. Existing benchmarks often rely on structured sources such as Wikidata, where changes are directly exposed through versioned snapshots, and rarely account for differences in model knowledge cut-offs, limiting fair evaluation across LLMs released at different times. We introduce a novel framework for automatically constructing temporal knowledge probing benchmarks from unstructured, time-stamped corpora by discovering evolving knowledge through salient entities, concepts, and attributes; and using this structure to generate grounded questions with gold answers aligned to model cut-off dates. Using this framework, we construct **evolveQA**, spanning 3 domains: AWS service updates, Azure platform changes, and WHO Disease Outbreak News reports. Evaluating 12 open- and closed-source LLMs across 3 probing formats, we find performance drops of up to 31% on **evolveQA** compared to static knowledge questions, showing that LLMs struggle to recall temporally updated facts even when such knowledge is available in their parameters.