Jichen Chai
Author directory2026
Latent Visual Sensitive Fingerprinting for Black-Box Vision–Language Model Tamper Detection
Chaoxiang He | Jichen Chai | Hanqing Hu | Yi Wang | Shuo Wang | Hongsheng Hu | Bin Benjamin Zhu
Findings of the Association for Computational Linguistics: EMNLP 2026
Chaoxiang He | Jichen Chai | Hanqing Hu | Yi Wang | Shuo Wang | Hongsheng Hu | Bin Benjamin Zhu
Findings of the Association for Computational Linguistics: EMNLP 2026
Cloud-hosted vision-language models (VLMs) may be silently fine-tuned, poisoned, compressed, or replaced after deployment, motivating black-box fingerprint verification. RESF tests response distributions but requires estimating the service temperature, making its reliability sensitive to estimation error. We propose Latent Visual Sensitive Fingerprinting (LVSF), based on a local Pearson-sensitivity decomposition of the conditional top-k distribution of the first generated token. This decomposition separates observable output shifts from input-dependent exposure to parameter changes. Modeling tampering-induced relative top-k logit shifts as isotropic, we show that a uniform top-k anchor maximizes average sensitivity. Accordingly, LVSF equalizes recorded top-k logits, maintains a margin from out-of-support tokens, and maximizes full-parameter Fisher sensitivity, yielding a reference distribution invariant to positive temperature scaling. These principles apply to both LLMs and VLMs. For VLMs, LVSF fixes a natural-language query and optimizes an image in a pretrained VAE latent space to produce natural-looking fingerprints. Repeated API queries detect excess out-of-support first tokens or shifts in conditional in-support frequencies. Across four VLMs and multiple tampering types, LVSF achieves consistently high detection rates with no observed false positives and substantially higher detection rates than evaluated baselines on Qwen2.5-VL-7B-Instruct.