Large language models (LLMs) are increasingly applied to vulnerability detection in real-world software systems, where the task goes beyond binary classification and requires calling context exploration over large code repositories. However, existing benchmarks primarily formulate the vulnerability detection as a classification problem without calling contexts and provide no formalized explanations, making them inadequate for evaluating LLM-driven vulnerability detectors upon repositories. To fill the gap, we introduce ManBo, a benchmark designed to assess the capability of LLM-driven vulnerability detectors in the real-world scenarios. Specifically, ManBo provides human-verified vulnerability explanations together with calling contexts, enabling principled evaluation of the ability to both identify vulnerabilities and explain root causes. It contains 205 vulnerable instances and 164 safe instances collected from 74 high-profile C/C++ projects, covering three critical types of memory corruption. We evaluate a broad spectrum of LLM-driven detectors on ManBo, and find that effective calling context retrieval is essential for repository-level vulnerability detection, exposing a fundamental gap between the capabilities of existing LLM-driven detectors and real-world requirements. ManBo is publicly available at https://github.com/jinyaoguo/ManBo.

Research is a fundamental process driving the advancement of human civilization, yet it demands substantial time and effort from researchers. In recent years, the rapid development of artificial intelligence (AI) technologies has inspired researchers to explore how AI can accelerate and enhance research. To monitor relevant advancements, this paper presents a systematic review of the progress in this domain. Specifically, we organize the relevant studies into three main categories: hypothesis formulation, hypothesis validation, and manuscript publication. Hypothesis formulation involves knowledge synthesis and hypothesis generation. Hypothesis validation includes the verification of scientific claims, theorem proving, and experiment validation. Manuscript publication encompasses manuscript writing and the peer review process. Furthermore, we identify and discuss the current challenges faced in these areas, as well as potential future directions for research. Finally, we also offer a comprehensive overview of existing benchmarks and tools across various domains that support the integration of AI into the research process. We hope this paper serves as an introduction for beginners and fosters future research.