Zhou Xuan
Author directory2026
Benchmarking Explainable Vulnerability Detection with Calling Contexts
Jinyao Guo | Yaoyang Ye | Zhihao Guo | Mingwei Zheng | Tiantai Zhang | Xuwei Liu | Xiaolong Jin | Congyu Liu | Xiangzhe Xu | Zian Su | Ziyun Song | Zhou Xuan | Xiao Cheng | Chengpeng Wang | Xiangyu Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Jinyao Guo | Yaoyang Ye | Zhihao Guo | Mingwei Zheng | Tiantai Zhang | Xuwei Liu | Xiaolong Jin | Congyu Liu | Xiangzhe Xu | Zian Su | Ziyun Song | Zhou Xuan | Xiao Cheng | Chengpeng Wang | Xiangyu Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) are increasingly applied to vulnerability detection in real-world software systems, where the task goes beyond binary classification and requires calling context exploration over large code repositories. However, existing benchmarks primarily formulate the vulnerability detection as a classification problem without calling contexts and provide no formalized explanations, making them inadequate for evaluating LLM-driven vulnerability detectors upon repositories. To fill the gap, we introduce ManBo, a benchmark designed to assess the capability of LLM-driven vulnerability detectors in the real-world scenarios. Specifically, ManBo provides human-verified vulnerability explanations together with calling contexts, enabling principled evaluation of the ability to both identify vulnerabilities and explain root causes. It contains 205 vulnerable instances and 164 safe instances collected from 74 high-profile C/C++ projects, covering three critical types of memory corruption. We evaluate a broad spectrum of LLM-driven detectors on ManBo, and find that effective calling context retrieval is essential for repository-level vulnerability detection, exposing a fundamental gap between the capabilities of existing LLM-driven detectors and real-world requirements. ManBo is publicly available at https://github.com/jinyaoguo/ManBo.