2026

Long-memory benchmarks usually ask whether an agent can recover evidence from prior dialogue and answer questions about it. Persistent assistants also face governed-use decisions: a remembered fact may later be revoked, superseded, restricted to a private context, or irrelevant to the current request. We introduce TimlMemBench, a diagnostic benchmark for governed use of long-term memory in dialogue agents. It starts from reviewed raw sessions and anchors each derived memory asset back to source evidence. The current package contains 208 reviewed Chinese and English samples, 1,566 sessions, 1,715 questions, and 1,821 governance or activation cases. TimlMemBench reports two complementary evaluations. The canonical-pool LLM-QA control compares systems over the same sample-local memory pool. The raw-native evaluation requires external memory systems to ingest raw sessions before answering governed-memory cases. Experiments show that memory-equipped systems reach high question-answering accuracy in the canonical-pool control, but their raw-native behavior is less stable on lifecycle updates, scope and boundary control, provenance recovery, activation, and restraint. TimlMemBench therefore exposes memory failures that are invisible to retrieval-oriented or answer-accuracy-only evaluation.

2024

In Visually-rich Document Understanding (VrDU), recent advances of incorporating layout and image features into the pre-training language models have achieved significant progress. Existing methods usually developed complicated dedicated architectures based on pre-trained models and fine-tuned them with costly high-quality data to eliminate the inconsistency of knowledge distribution between the pre-training task and specialized downstream tasks. However, due to their huge data demands, these methods are not suitable for few-shot settings, which are essential for quick applications with limited resources but few previous works are presented. To solve these problems, we propose a unified Knowledge-aware prompt-tuning framework for Visual-rich Document Understanding (KnowVrDU) to enable broad utilization for diverse concrete applications and reduce data requirements. To model heterogeneous VrDU structures without designing task-specific architectures, we propose to reformulate various VrDU tasks into a single question-answering format with task-specific prompts and train the pre-trained model with the parameter-efficient prompt tuning method. To bridge the knowledge gap between the pre-training task and specialized VrDU tasks without additional annotations, we propose a prompt knowledge integration mechanism to leverage external open-source knowledge bases. We conduct experiments on several benchmark datasets in few-shot settings and the results validate the effectiveness of our method.