NEXUS: Structured Runtime Safety for Tool-Using LLM Agent

Elias Hossain, Md Mehedi Hasan Nipu, Tasfia Nuzhat Ornee, Rajib Rana, Niloofar Yousefi


Abstract
Tool-using LLM agents increasingly execute high-impact actions, making runtime safety monitoring essential. We present NEXUS (Neural EXecution Utility and Safety), a structured-plan safety monitor that applies a formal intervention policy Π to select among four actions: allow, block, request confirmation, or request revision. NEXUS combines deterministic safety rules, argument-level inspection, and a calibrated logistic-regression risk score for graded escalation. On a 128-instance synthetic benchmark, NEXUS attains F1 = 0.949 and 4-class intervention accuracy of 0.6406, outperforming rule-only intervention selection by +27.3 percentage points. It also improves over rule-only on R-Judge (F1 = 0.861 vs. 0.849), matches rule-only on AgentHarm due to threat-model limits, and achieves 0% ASR at 99% control allow on IPI. On the rule-blind NEXUS-Stress benchmark, NEXUS reaches F1 = 0.897 with REVISE recall of 0.500, highlighting both progress and remaining difficulty in fine-grained intervention routing. With 0.38 ms median latency, NEXUS adds under 0.1% overhead to typical agent loops.
Anthology ID:
2026.findings-aacl.112
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2203–2232
Language:
URL:
https://aclanthology.org/2026.findings-aacl.112/
DOI:
Bibkey:
Cite (ACL):
Elias Hossain, Md Mehedi Hasan Nipu, Tasfia Nuzhat Ornee, Rajib Rana, and Niloofar Yousefi. 2026. NEXUS: Structured Runtime Safety for Tool-Using LLM Agent. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 2203–2232, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
NEXUS: Structured Runtime Safety for Tool-Using LLM Agent (Hossain et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.112.pdf