Nicole Hu
Author directoryAlso published as: Nicole HU
2026
A Survey of Secure Retrieval-Augmented Generation
Yuming XU | Mingtao Zhang | Zhuohan Ge | Haoyang LI | Nicole HU | Yongqi Zhang | Zhiyuan Wen | Chen Jason Zhang | Li Qing | Lei Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Yuming XU | Mingtao Zhang | Zhuohan Ge | Haoyang LI | Nicole HU | Yongqi Zhang | Zhiyuan Wen | Chen Jason Zhang | Li Qing | Lei Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Retrieval-augmented generation (RAG) improves large language models (LLMs) with external knowledge, but this access path creates security risks distinct from inherent prompt-only or parametric-model flaws. We frame secure RAG as securing external knowledge access. We conducted a systematic search and curated 135 works on attacks, defenses, and security evaluation, and organized them with SLOT: a taxonomy along the attack Surface (S) and the corresponding defense Layer (L), with cross-cutting axes Objective (O) and attack-target scope (T). Mapping these studies onto an external knowledge-access pipeline, we expose three mismatches: target mismatch (T2 attacks outpace T2 defenses and evaluation), stage mismatch (S1 attacks outnumber L1 defenses), and signal mismatch (fluent, retrievable, corpus-fitting S1 attacks challenge anomaly-based L2/L3 defenses). Finally, we discuss directions for more realistic targets, surface-complete defense stacks, standardized evaluation, confidentiality, and multimodal and agentic systems, and release the screening process, selected-paper metadata, and machine-readable SLOT labels at https://github.com/TreeAI-Lab/Awesome-RAG-Security.
2025
Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Models
Haoyang Li | Xuejia Chen | Zhanchao Xu | Darian Li | Nicole Hu | Fei Teng | Yiming Li | Luyu Qiu | Chen Jason Zhang | Li Qing | Lei Chen
Findings of the Association for Computational Linguistics: ACL 2025
Haoyang Li | Xuejia Chen | Zhanchao Xu | Darian Li | Nicole Hu | Fei Teng | Yiming Li | Luyu Qiu | Chen Jason Zhang | Li Qing | Lei Chen
Findings of the Association for Computational Linguistics: ACL 2025
Large Language Models (LLMs) have demonstrated impressive capabilities in natural language processing tasks, such as text generation and semantic understanding. However, their performance on numerical reasoning tasks, such as basic arithmetic, numerical retrieval, and magnitude comparison, remains surprisingly poor. This gap arises from their reliance on surface-level statistical patterns rather than understanding numbers as continuous magnitudes. Existing benchmarks primarily focus on either linguistic competence or structured mathematical problem-solving, neglecting fundamental numerical reasoning required in real-world scenarios. To bridge this gap, we propose NumericBench, a comprehensive benchmark to evaluate six fundamental numerical capabilities: number recognition, arithmetic operations, contextual retrieval, comparison, summary, and multi-step reasoning. NumericBench includes datasets ranging from synthetic number lists to crawled real-world data, addressing challenges like long contexts, noise, and multi-step reasoning. Extensive experiments on state-of-the-art LLMs, including GPT-4 and DeepSeek, reveal persistent weaknesses in numerical reasoning, highlighting the urgent need to improve numerically-aware language modeling. The benchmark is released in: https://github.com/TreeAI-Lab/NumericBench.