Beiya Dai
Author directory2026
DoPR: Reusable Compressed Document Prefixes for Efficient LLM Reranking
Beiya Dai | Yifan Wei | Guang Yang | Xing Shi | Xinbing Wang | Zhouhan Lin
Findings of the Association for Computational Linguistics: EMNLP 2026
Beiya Dai | Yifan Wei | Guang Yang | Xing Shi | Xinbing Wang | Zhouhan Lin
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) are effective rerankers, but pointwise reranking repeatedly processes the same document across different queries, causing substantial redundant document-side computation. We propose DoPR, a compressed document prefix framework that decouples offline document processing from online reranking. DoPR first selects query-independent document representations and converts them into compressed document prefix states, which are precomputed offline and reused whenever the document is retrieved. During online reranking, the model scores each query-document pair by processing only the query and scoring token, with document information supplied by the stored prefix states. This design reduces online cost through both document-side compression and cross-query prefix-state reuse. Experiments on TREC DL, BEIR, and BRIGHT with Qwen3 models from 0.6B to 8B show that DoPR achieves up to 8.0× online document-side memory reduction and up to 8.04× latency speedup, while retaining 97.1%-99.5% of the average NDCG@10 of matched full-document rerankers.