Yongting Zhang
Author directory2026
An Automated Pipeline for Provably Retrieval-Dependent Benchmark Construction over Dynamic Knowledge
Heng Zhou | Ao Yu | Yuchen Fan | Li Kang | Jianing Shi | Yongting Zhang | Yutao Fan | Tiancheng He | Yibing Lin | Hejia Geng | Yuhao Wu | Xiufeng Song | Zhemeng Zhang | Songtao Huang | Yiran Qin | Wenlong Zhang | Lei Bai | Zhenfei Yin
Findings of the Association for Computational Linguistics: EMNLP 2026
Heng Zhou | Ao Yu | Yuchen Fan | Li Kang | Jianing Shi | Yongting Zhang | Yutao Fan | Tiancheng He | Yibing Lin | Hejia Geng | Yuhao Wu | Xiufeng Song | Zhemeng Zhang | Songtao Huang | Yiran Qin | Wenlong Zhang | Lei Bai | Zhenfei Yin
Findings of the Association for Computational Linguistics: EMNLP 2026
Static benchmarks often conflate memorization with reasoning, failing to capture the dynamic nature of world knowledge. We present LiveSearchBench, a scalable pipeline that automatically constructs snapshot-grounded benchmarks from knowledge graph differentials, in which every answer is unique and SPARQL-verifiable against a target snapshot. Unlike prior template-based approaches, our method leverages topology-aware synthesis to generate multi-constraint compositional questions with unique answers guaranteed via SPARQL validation. Evaluating 1,000 questions across three difficulty levels on both open-weight and frontier models, we expose a pronounced Recency Gap on facts that post-date a model’s training cutoff, particularly for compositional queries; retrieval dependence is thus an empirical property we measure, not a formal guarantee we claim. Through Oracle and Wikidata-as-corpus analyses, we disentangle dual failure modes: retrieval coverage limitations for single-hop questions and reasoning failures for compositional queries even with ground-truth evidence. LiveSearchBench provides both a rigorous construction methodology and a continuously updatable testbed for evaluating evidence integration under evolving knowledge.
2024
PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety
Zaibin Zhang | Yongting Zhang | Lijun Li | Hongzhi Gao | Lijun Wang | Huchuan Lu | Feng Zhao | Yu Qiao | Jing Shao
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zaibin Zhang | Yongting Zhang | Lijun Li | Hongzhi Gao | Lijun Wang | Huchuan Lu | Feng Zhao | Yu Qiao | Jing Shao
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Multi-agent systems, when enhanced with Large Language Models (LLMs), exhibit profound capabilities in collective intelligence. However, the potential misuse of this intelligence for malicious purposes presents significant risks. To date, comprehensive research on the safety issues associated with multi-agent systems remains limited. In this paper, we explore these concerns through the innovative lens of agent psychology, revealing that the dark psychological states of agents constitute a significant threat to safety.To tackle these concerns, we propose a comprehensive framework (PsySafe) grounded in agent psychology, focusing on three key areas: firstly, identifying how dark personality traits in agents can lead to risky behaviors; secondly, evaluating the safety of multi-agent systems from the psychological and behavioral perspectives, and thirdly, devising effective strategies to mitigate these risks.Our experiments reveal several intriguing phenomena, such as the collective dangerous behaviors among agents, agents’ self-reflection when engaging in dangerous behavior, and the correlation between agents’ psychological assessments and dangerous behaviors. We anticipate that our framework and observations will provide valuable insights for further research into the safety of multi-agent systems. We make our data and code publicly accessible at https://github.com/AI4Good24/PsySafe.