Vista: An Efficient Late Interaction Engine for Multi-Vector Visual Document Retrieval

Haolin Jiang, Jinwen Pan, Abhishek Saini, Weihang Li, Hang Liu


Abstract
Visual document retrieval (VDR) bridges language-based systems with visually grounded knowledge. Recent VDR models adopt multi-vector late interaction, representing each document with multiple visual token embeddings and scoring relevance with MaxSim. However, existing late interaction engines struggle to support multi-vector VDR at scale due to the higher dimensionality and larger number of visual tokens per document.

We propose VISTA, a VDR-specific late-interaction engine that exploits the correlation among visual tokens within each document to optimize both compression and search. To the best of our knowledge, VISTA is the first VDR-specific late-interaction engine for scalable multi-vector visual document retrieval. Experiments on ViDoRe V2 and ViDoRe V3 show that VISTA achieves comparable or even better retrieval quality than compressed baselines, while consistently reducing latency by 1.2–6.7× and index size by 1.4–6.1×.
Anthology ID:
2026.emnlp-main.2135
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
48529–48543
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2135/
DOI:
Bibkey:
Cite (ACL):
Haolin Jiang, Jinwen Pan, Abhishek Saini, Weihang Li, and Hang Liu. 2026. Vista: An Efficient Late Interaction Engine for Multi-Vector Visual Document Retrieval. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 48529–48543, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Vista: An Efficient Late Interaction Engine for Multi-Vector Visual Document Retrieval (Jiang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2135.pdf