Zhaoxiang Wang

Author directory

2026

Large Language Models (LLMs) integrated into complex ecosystems (e.g., code agents) face critical safety threats from Indirect Prompt Injection (IPI) attacks, where malicious instructions are embedded in retrieved data. Existing defenses face an intractable safety-utility trade-off: sanitization-based guardrails incur high latency, while detection-based guardrails suffer from severe over-refusal. We first demonstrate, both theoretically and empirically, that LLMs can intrinsically separate instructions from data. Inspired by this insight, we propose AEGIS (*A*daptive *E*nsemble *G*uard for *I*njection *S*hielding), a training-free defense that extracts instruction-sensitive projectors to identify malicious instructions and leverages a Unified Multi-Layer Consensus that aggregates topologically distinct signals across the network depth. Evaluations on eight IPI attacks and ten baselines show that AEGIS achieves state-of-the-art detection accuracy while reducing the false positive rate on benign tasks to below 1%. Code is available at [Github](https://github.com/xaddwell/AEGIS).
Composed image retrieval (CIR) requires complex reasoning over heterogeneous visual and textual constraints. Existing approaches largely fall into two paradigms: unified embedding retrieval, which suffers from single-model myopia, and heuristic agentic retrieval, which is limited by suboptimal, trial-and-error orchestration. To this end, we propose OSCAR, an optimization-steered agentic planning framework for composed image retrieval. We are the first to reformulate agentic CIR from a heuristic search process into a principled trajectory optimization problem. Instead of relying on heuristic trial-and-error exploration, OSCAR employs a novel offline-online paradigm. In the offline phase, we model CIR via atomic retrieval selection and composition as a two-stage mixed-integer programming problem, mathematically deriving optimal trajectories that maximize ground-truth coverage for training samples via rigorous boolean set operations. These trajectories are then stored in a golden library to serve as in-context demonstrations for online steering of VLM planner at online inference time. Extensive experiments on three public benchmarks and a private industrial benchmark show that OSCAR consistently outperforms SOTA baselines. Notably, it achieves superior performance using only 10% of training data, demonstrating strong generalization of planning logic rather than dataset-specific memorization.