Who Speaks for the Pruned? Visual Token Pruning as Coverage Optimization

Qingchan Zhu, Weihang You, Hanqi Jiang, Changdi Yang, Tianming Liu, Geng Yuan


Abstract
Visual token pruning reduces the inference cost of vision-language models (VLMs), but most methods only ask which tokens to keep. This retained-token view can keep redundant high-scoring tokens while leaving discarded evidence without a close representative. We propose CoverPruner, a training-free pruner that asks the complementary demand-side question: after a token is removed, which surviving original token represents it for the target VLM? CoverPruner formulates pruning as Representational Coverage Maximization (RCM), covering the full projected visual-token set with query-weighted demand. It instantiates RCM with projector-space coverage and a lightweight first-layer attention probe. Across multiple VLM architectures and compression rates, CoverPruner achieves the best average accuracy among all compared methods, with the largest gains usually appearing under aggressive compression.
Anthology ID:
2026.emnlp-main.349
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
8018–8037
Language:
URL:
https://aclanthology.org/2026.emnlp-main.349/
DOI:
Bibkey:
Cite (ACL):
Qingchan Zhu, Weihang You, Hanqi Jiang, Changdi Yang, Tianming Liu, and Geng Yuan. 2026. Who Speaks for the Pruned? Visual Token Pruning as Coverage Optimization. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 8018–8037, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Who Speaks for the Pruned? Visual Token Pruning as Coverage Optimization (Zhu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.349.pdf