Guijia Zhang
Author directory2026
STARS: Skill Triage via Activation Risk Scoring for LLM Agents
Guijia Zhang | Shu Yang | Xilin Gong | Di Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Guijia Zhang | Shu Yang | Xilin Gong | Di Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
The risk of an agent skill call depends not on the skill alone but on the invocation context: the same file-reading skill is benign when listing logs, yet dangerous when tainted content steers the agent toward secret exfiltration. We formalize invocation-time auditing: intercept a proposed skill call before execution and score its risk from the user request, the candidate skill, and the runtime context. STARS implements this as a three-stage pipeline that fuses a static capability prior with a request-conditioned invocation scorer. The primary configuration, STARS-LLM, uses an LLM judge; STARS-Rules is a deterministic, API-free alternative. On SIA-Bench, a 3,000-record synthetic benchmark with a 180-record human check at Fleiss κ = 0.709, request-conditioned scoring improves high-risk retrieval on held-out indirect prompt injection, where static metadata cannot see the attack. On a frozen 500/100 scoring subset, STARS-LLM raises rules-only HR-AUPRC from 0.297 to 0.728 in-distribution and from 0.504 to 0.757 on held-out IPI. That gain is in ranking: under frozen thresholds, held-out classification accuracy does not improve. SIA-Bench is a ranking stress test, not an estimate of real-world prevalence.
Hallucination as Exploit: Evidence-Carrying Multimodal Agents
Guijia Zhang | Hao Zheng | Harry Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Guijia Zhang | Hao Zheng | Harry Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Multimodal agents increasingly choose tool calls from screenshots, documents, and webpages, where a false perceptual claim can turn hallucination from an answer-quality error into an authorization failure. We formalize this failure as hallucination-to-action conversion (H2AC) and propose evidence-carrying multimodal agents (ECA), in which model text may propose actions but cannot authorize them: constrained DOM/OCR/AX verifiers issue typed certificates for action-critical predicates, and a deterministic gate executes only what the certificates support. Across verifier red-teaming, 7,488 replayed authorization traces, and end-to-end and live-browser evaluations, ECA executes no unsafe action, while GPT-5.4 judge baselines admit 79–99% of unsafe actions under the same threat model; benign actions are occasionally deferred to user confirmation (6.4% on live pages) but never hard-blocked. The guarantee is conditional, extending exactly as far as the verifiers, schemas, and gate implementation that form ECA’s trusted base.