Dun Zhang
Author directory2026
Jasper-Flash: Elastic Token Compression and Progressive Distillation for Inference-Scalable Text Embedding Models
Haining An | Xingbing Chen | Wenbo Lyu | Dun Zhang | Ziyang Zeng | Zhenxiang Zhao | Rui Song | Hao Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Haining An | Xingbing Chen | Wenbo Lyu | Dun Zhang | Ziyang Zeng | Zhenxiang Zhao | Rui Song | Hao Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Deploying text embedding models under resource constraints is challenged by the sequence-length-dependent computational cost of self-attention. However, existing sequence-reduction strategies remain predominantly static and offer limited flexibility across inference budgets. We propose Jasper-Flash, an inference-scalable text embedding framework combining Elastic Token Compression (ETC) with Compression-Adaptive Progressive Distillation (CAPD). ETC uses a length threshold and a user-selected compression ratio to adjust sequence length on the fly, enabling flexible latency–quality trade-offs within a single model. To mitigate the optimization difficulty introduced by lossy sequence reduction, CAPD integrates heterogeneous multi-teacher distillation with a progressive curriculum over fixed and dynamically sampled compression ratios. We instantiate Jasper-Flash as Jasper-Token-Compression-600M, a bilingual text embedding model. Experiments on MTEB and C-MTEB demonstrate competitive performance across compression ratios and substantial reductions in encoding latency, particularly for longer inputs. The code and model are available at https://huggingface.co/infgrad/Jasper-Token-Compression-600M.
2025
An Empirical Study of Position Bias in Modern Information Retrieval
Ziyang Zeng | Dun Zhang | Jiacheng Li | Zoupanxiang | Yudong Zhou | Yuqing Yang
Findings of the Association for Computational Linguistics: EMNLP 2025
Ziyang Zeng | Dun Zhang | Jiacheng Li | Zoupanxiang | Yudong Zhou | Yuqing Yang
Findings of the Association for Computational Linguistics: EMNLP 2025
This study investigates the position bias in information retrieval, where models tend to overemphasize content at the beginning of passages while neglecting semantically relevant information that appears later. To analyze the extent and impact of position bias, we introduce a new evaluation framework consisting of two position-aware retrieval benchmarks (SQuAD-PosQ, FineWeb-PosQ) and an intuitive diagnostic metric, the Position Sensitivity Index (PSI), for quantifying position bias from a worst-case perspective. We conduct a comprehensive evaluation across the full retrieval pipeline, including BM25, dense embedding models, ColBERT-style late-interaction models, and full-interaction reranker models. Our experiments show that when relevant information appears later in the passage, dense embedding models and ColBERT-style models suffer significant performance degradation (an average drop of 15.6%). In contrast, BM25 and reranker models demonstrate greater robustness to such positional variation. These findings provide practical insights into model sensitivity to the position of relevant information and offer guidance for building more position-robust retrieval systems. Code and data are publicly available at: https://github.com/NovaSearch-Team/position-bias-in-IR.