Jiaming Zhang
Author directoryOther people with similar names: Jiaming Zhang, Jiaming Zhang
Unverified author pages with similar names: Jiaming Zhang
2026
AEMAgent: Cognition-Inspired Active Memory Formation for Very-Long Video Understanding
Xian Zhao | Jiaming Zhang | Haonan Chu | Weihao Bo | Xu Chen | Delai Qiu | Yining Wang | Yi Zhang | Shengping Liu | Jian Yu | Jitao Sang
Findings of the Association for Computational Linguistics: EMNLP 2026
Xian Zhao | Jiaming Zhang | Haonan Chu | Weihao Bo | Xu Chen | Delai Qiu | Yining Wang | Yi Zhang | Shengping Liu | Jian Yu | Jitao Sang
Findings of the Association for Computational Linguistics: EMNLP 2026
Embodied AI and wearable assistants require agents to retrieve sparse evidence from continuous days-level videos. Existing memory-based methods typically follow a passive recording paradigm based on fixed-window captioning and periodic memory construction, which produces redundant memories and inefficient long-horizon reasoning. Inspired by human memory formation, we propose Active Explore Memory Agent (AEMAgent), which transforms passive recording into active memory formation by deciding when to remember, how to explore events, and when to reflect. These decisions are instantiated as a coherent pipeline: Surprise-guided Event Selection detects salient event changes for selective memory writing, Exploratory Episodic Memory Encoding enriches accepted events through self-questioning, Predictive-Coding-Guided Reflection updates knowledge from expectation-observation mismatches, and Semantic Memory Consolidation distills noisy episodic updates into compact long-term memory. Experiments on multiple benchmarks show that AEMAgent achieves the best overall performance among all compared methods while maintaining lower storage overhead than strong full-memory baselines, demonstrating the effectiveness of selective and efficient memory formation for scalable very-long video understanding.
GUITester: Enabling GUI Agents for Exploratory Defect Discovery
Yifei Gao | Jiang Wu | Xiaoyi Chen | Yifan Yang | Zhe Cui | Tianyi Ma | Jiaming Zhang | Jitao Sang
Findings of the Association for Computational Linguistics: ACL 2026
Yifei Gao | Jiang Wu | Xiaoyi Chen | Yifan Yang | Zhe Cui | Tianyi Ma | Jiaming Zhang | Jitao Sang
Findings of the Association for Computational Linguistics: ACL 2026
Exploratory GUI testing is essential for software quality but suffers from high manual costs. While Multi-modal Large Language Model (MLLM) agents excel in navigation, they fail to autonomously discover defects due to two core challenges: Goal-Oriented Masking, where agents prioritize task completion over reporting anomalies, and Execution-Bias Attribution, where system defects are misidentified as agent errors. To address these, we first introduce GUITestBench, the first interactive benchmark for this task, featuring 143 tasks across 26 defects. We then propose GUITester, a multi-agent framework that decouples navigation from verification via two modules: (i) a Planning-Execution Module (PEM) that proactively probes for defects via embedded testing intents, and (ii) a Hierarchical Reflection Module (HRM) that resolves attribution ambiguity through interaction history analysis. GUITester achieves an F1-score of 48.90% (Pass@3) on GUITestBench, outperforming state-of-the-art baselines (33.35%). Our work demonstrates the feasibility of autonomous exploratory testing and provides a robust foundation for future GUI quality assurance.
AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Che Wang | Jiaming Zhang | Ziqi Zhang | Zijie Wang | Yinggui Wang | Jianbo Gao | Tao Wei | Zhong Chen | Wei Yang Bryan Lim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Che Wang | Jiaming Zhang | Ziqi Zhang | Zijie Wang | Yinggui Wang | Jianbo Gao | Tao Wei | Zhong Chen | Wei Yang Bryan Lim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
As open-source large language models (LLMs) become increasingly powerful, they are rapidly evolving into Agentic LLMs capable of automating complex daily tasks. However, this paradigm shift entails severe security risks, as the agents’ necessary integration with external real-time data sources creates a direct pathway for indirect prompt injection (IPI) attacks. Existing attack methods are limited by their reliance on static patterns and evaluation on simple language models, failing to address the fast-evolving nature of modern Agentic LLMs. We introduce AdapTools, a novel adaptive IPI attack framework that selects stealthier attack tools and generates adaptive attack prompts to create a rigorous security evaluation environment. Our approach comprises two key components: (1) Adaptive Attack Strategy Construction, which develops transferable adversarial strategies for prompt optimization, and (2) Attack Enhancement, which identifies stealthy tools capable of circumventing task-relevance defenses. Comprehensive experiments on both open-source and commercial agentic LLMs show that AdapTools achieves a per-model average improvement of 2.08× in attack success rate over the strongest baseline under state-of-the-art defenses (MELON, Pi-Detector), with gains reaching up to 3.06× on the most resistant reasoning agents. The framework further transfers to black-box commercial agents (GPT, Qwen, Gemini), with ASR ranging from 4.2 to 20.3 (peak on the in-family Qwen-MAX), exposing systematic vulnerabilities in modern Agentic LLMs.
2025
Investigating and Enhancing Vision-Audio Capability in Omnimodal Large Language Models
Rui Hu | Delai Qiu | Shuyu Wei | Jiaming Zhang | Yining Wang | Shengping Liu | Jitao Sang
Findings of the Association for Computational Linguistics: ACL 2025
Rui Hu | Delai Qiu | Shuyu Wei | Jiaming Zhang | Yining Wang | Shengping Liu | Jitao Sang
Findings of the Association for Computational Linguistics: ACL 2025
Omnimodal Large Language Models (OLLMs) have shown significant progress in integrating vision and text, but still struggle with integrating vision and audio, often exhibiting suboptimal performance when processing audio queries compared to text queries. This disparity is primarily due to insufficient alignment between vision and audio modalities during training, leading to inadequate attention to visual information when using audio queries. To mitigate this issue, we propose a Self-Knowledge Distillation (Self-KD) training method where the vision-text component of the OLLM serves as the teacher and the vision-audio component as the student. This enables the model to process audio in a manner analogous to its text processing. Our experimental results demonstrate that Self-KD is an effective method for enhancing the vision-audio capabilities of OLLMs by learning from the vision-text components, which subsequently improves the interaction between audio and images and results in improved performance on multimodal tasks.