Persistent AI assistants, such as OpenClaw, accumulate large collections of related memories over long-term interactions. As these memories grow, they may reinforce one another, diverge across contexts, or directly conflict, making correct assistance depend on memory relations rather than isolated recall. Existing long-term memory benchmarks do not systematically probe how agents preserve and utilize such relations during downstream tasks. To address this gap, we introduce SubtleMemory, a benchmark for fine-grained relational memory discrimination in long-running AI agents. SubtleMemory constructs relation-controlled latent semantic artifacts whose variants instantiate complementary, nuanced, or contradictory relations, and embeds them into realistic user-agent histories, requiring agents to recover distributed relational structures during later queries and instructions.The benchmark contains 1,522 evaluation instances over 10 long histories, grounded in 1,090 relation-controlled memory-variant sets and spanning user-related and non-user-related queries. Evaluating six standalone memory systems, two Claw-style agents with native memory modules, and three Claw-style agents with plugin memory modules, we find that current systems remain weak on fine-grained relational memory discrimination. We further introduce diagnostic protocols that reveal distinct capability profiles across memory preservation, retrieval, and downstream reasoning stages.

Planning is central to LLM agents: before acting, an agent must decompose goals, select tools, reason over constraints, and decide when a task is infeasible. Yet existing agent evaluations often report only end-to-end success, making it difficult to determine whether failures stem from planning or execution. We introduce Agent Planning Benchmark (APB), a planning-specific diagnostic benchmark with 4,209 multimodal cases across 22 domains and five settings, covering holistic planning, feedback-conditioned step-wise planning, and robustness under extraneous tools, broken tools, and unsolvable tasks. Across 12 MLLMs, APB reveals systematic weaknesses in long-horizon planning, tool-noise robustness, calibrated refusal, and inference-time refinement. We further validate APB on 200 ToolSandbox tasks and 200 𝜏²-bench tasks, where APB-guided refinement consistently improves plan correctness, plan grade, and downstream execution metrics across three representative models. APB thus serves as an upstream diagnostic complement to execution benchmarks. The APB benchmark and code are available in this URL.

We present FullFront, a workflow-grounded benchmark for Multimodal Large Language Models (MLLMs) across the front-end engineering pipeline—conceptualization, comprehension, and implementation—rather than evaluating any single stage in isolation. FullFront contributes along three axes: (i) a unified task formulation covering Webpage Design, Webpage Perception QA, and Webpage Code Generation (8 subtasks, 2,700 items); (ii) a controlled two-stage data construction pipeline that converts real-world webpages into standardized HTML with a 15-category copyright-safe image substitution scheme; and (iii) a reliability-oriented evaluation framework whose Code Score and Gemini Visual Score are validated against human judgment, cross-judge agreement, pipeline bias, and sensitivity stress tests. Across 12 MLLMs we observe a stable closed-source top tier, a Perception–Implementation Gap with distinct failure modes (Coding Bottleneck” for open-source, Grounding Failure” for proprietary), and heterogeneous open/closed capability profiles—patterns that isolated benchmarks cannot expose.