Latent Visual Sensitive Fingerprinting for Black-Box Vision–Language Model Tamper Detection

Chaoxiang He, Jichen Chai, Hanqing Hu, Yi Wang, Shuo Wang, Hongsheng Hu, Bin Benjamin Zhu


Abstract
Cloud-hosted vision-language models (VLMs) may be silently fine-tuned, poisoned, compressed, or replaced after deployment, motivating black-box fingerprint verification. RESF tests response distributions but requires estimating the service temperature, making its reliability sensitive to estimation error. We propose Latent Visual Sensitive Fingerprinting (LVSF), based on a local Pearson-sensitivity decomposition of the conditional top-k distribution of the first generated token. This decomposition separates observable output shifts from input-dependent exposure to parameter changes. Modeling tampering-induced relative top-k logit shifts as isotropic, we show that a uniform top-k anchor maximizes average sensitivity. Accordingly, LVSF equalizes recorded top-k logits, maintains a margin from out-of-support tokens, and maximizes full-parameter Fisher sensitivity, yielding a reference distribution invariant to positive temperature scaling. These principles apply to both LLMs and VLMs. For VLMs, LVSF fixes a natural-language query and optimizes an image in a pretrained VAE latent space to produce natural-looking fingerprints. Repeated API queries detect excess out-of-support first tokens or shifts in conditional in-support frequencies. Across four VLMs and multiple tampering types, LVSF achieves consistently high detection rates with no observed false positives and substantially higher detection rates than evaluated baselines on Qwen2.5-VL-7B-Instruct.
Anthology ID:
2026.findings-emnlp.2178
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
46570–46589
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2178/
DOI:
Bibkey:
Cite (ACL):
Chaoxiang He, Jichen Chai, Hanqing Hu, Yi Wang, Shuo Wang, Hongsheng Hu, and Bin Benjamin Zhu. 2026. Latent Visual Sensitive Fingerprinting for Black-Box Vision–Language Model Tamper Detection. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 46570–46589, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Latent Visual Sensitive Fingerprinting for Black-Box Vision–Language Model Tamper Detection (He et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2178.pdf