Zhisheng Chen

Author directory

2026

Persistent LLM agents require memory representations that make the formation of person understanding explicit across long term interaction. Existing agent memory methods emphasize information retention and retrieval, yet give limited account of how accumulated interaction evidence is abstracted into person understanding. We view this process as schema formation, where situated evidence is abstracted into reusable patterns and stable person level claims. We introduce PersonaTree, a structured lifecycle memory framework that realizes this view as a three level persona tree with explicit support paths from evidence to claims. PersonaTree maintains the tree through conservative writing, confidence guided consolidation, and query conditioned path retrieval, returning only the evidence depth required by each query. Across six person understanding and persistent memory benchmarks with three answer backbones, PersonaTree ranks first in 12 of 18 compact scores and reaches the top two in 16 settings. Ablations show that hierarchy improves abstract person understanding on KnowMe, while support path retrieval improves RealPref alignment under a comparable context budget.
The stateless architecture of Large Language Models inherently lacks the mechanism to preserve dynamic context, compelling agents to redundantly reprocess history to maintain long-horizon autonomy. While latent memory offers a solution, current approaches are hindered by architectural segregation, relying on auxiliary encoders that decouple memory from the reasoning backbone. We propose FlashMem, a framework that distills intrinsic memory directly from transient reasoning states via computation reuse. Leveraging the property that internal representations uniquely encode input trajectories, FlashMem identifies the last hidden state as a sufficient statistic for the interaction history. This enables a Shared-KV Consolidator to synthesize memory by attending directly to the backbone’s frozen cache, eliminating redundant re-parameterization. Furthermore, a parameter-free Cognitive Monitor leverages attention entropy to adaptively trigger consolidation only when high epistemic uncertainty is detected. Experiments demonstrate that FlashMem matches the performance of heavy baselines while reducing inference latency by 5 times, effectively bridging the gap between efficiency and persistent cognition.
Questions about a recurring person in long videos are rarely answered from a single moment. They require evidence that is sufficient for the claim, comparisons across episodes, synthesis over recurring audiovisual patterns, and refusal when the source material does not support an answer. Existing long-video benchmarks do not isolate these requirements. We introduce KnowMeBenchV2, an evaluation-only benchmark over five public multi-episode video series, with 327 videos and 3,061 questions covering four evidence scopes: single-clip extraction, temporal and event relations, cross-episode ordering, and cross-episode persona synthesis. A separate grounded-abstention task serves as a contrastive support check: its questions are surface-similar to supported person-linked questions but are not licensed by the source evidence. The most interpretive synthesis items are human-authored and quality-audited by trained annotators with full source access. A matched human reference reaches 78.0 on cross-episode synthesis. To diagnose failures, we compare evidence routes under a fixed Qwen3.5-9B answerer and a task-matched pre-tokenization character routing budget. ASR-Retrieval reaches 2.7 on synthesis, while KnowMe reaches 64.9. In this diagnostic, the final answerer reasons over textualized evidence constructed from audiovisual sources rather than raw video tokens. The gap therefore isolates the importance of audiovisual evidence construction and cross-episode organization before answer generation.
Existing long-horizon memory benchmarks mostly use multi-turn dialogues or synthetic user histories, which makes retrieval performance an imperfect proxy for person understanding. We present Knowme-Bench, a publicly releasable benchmark built from long-form autobiographical narratives, where actions, context, and inner thoughts provide dense evidence for inferring stable motivations and decision principles. Knowme-Bench reconstructs each narrative into a flashback-aware, time-anchored stream and evaluates models with evidence-linked questions spanning factual recall, subjective state attribution, and principle-level reasoning. Across diverse narrative sources, retrieval-augmented systems mainly improve factual accuracy, while errors persist on temporally grounded explanations and higher-level inferences, highlighting the need for memory mechanisms beyond retrieval.