Weihang Li

Author directory

2026

Visual document retrieval (VDR) bridges language-based systems with visually grounded knowledge. Recent VDR models adopt multi-vector late interaction, representing each document with multiple visual token embeddings and scoring relevance with MaxSim. However, existing late interaction engines struggle to support multi-vector VDR at scale due to the higher dimensionality and larger number of visual tokens per document.

We propose VISTA, a VDR-specific late-interaction engine that exploits the correlation among visual tokens within each document to optimize both compression and search. To the best of our knowledge, VISTA is the first VDR-specific late-interaction engine for scalable multi-vector visual document retrieval. Experiments on ViDoRe V2 and ViDoRe V3 show that VISTA achieves comparable or even better retrieval quality than compressed baselines, while consistently reducing latency by 1.2–6.7× and index size by 1.4–6.1×.