Beyond Semantic IDs: Encoding Business-Value Ranking into Document Identifiers for Generative Retrieval

Gui Ling, Zhihong Chen, Yu Li, Tong Xiong, Kunhai Lin, Kaixuan Zhang, Yuliang Yan, Dan Ou, Tanghaihong, Bo Zheng


Abstract
Generative Retrieval (GR) formulates retrieval as a sequence-to-sequence generation task, assigning each document a document identifier (DocID) and retrieving it through autoregressive decoding, making DocID design a critical factor in retrieval quality. However, existing schemes based on discrete representation learning suffer from inherent collision issues and create a mismatch between the DocID’s encoding objective and the system’s business optimization target. To address these limitations, we propose Cluster-Ranked Identifier (CRID), which decouples DocID into semantic clustering and business-value ranking, yielding collision-free identifiers that support incremental updates via intra-cluster reranking. We further introduce an analytical framework that decomposes retrieval gains into personalized preference and statistical prior generalization, revealing how semantic cluster size governs the balance between the two components. Experiments on a Taobao e-commerce corpus of over 300M items show that CRID surpasses the strongest embedding-based retrieval baseline on top-K Hitrate, and delivers +1.06% GMV in full-traffic deployment.
Anthology ID:
2026.emnlp-industry.38
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
641–654
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.38/
DOI:
Bibkey:
Cite (ACL):
Gui Ling, Zhihong Chen, Yu Li, Tong Xiong, Kunhai Lin, Kaixuan Zhang, Yuliang Yan, Dan Ou, Tanghaihong, and Bo Zheng. 2026. Beyond Semantic IDs: Encoding Business-Value Ranking into Document Identifiers for Generative Retrieval. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 641–654, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Beyond Semantic IDs: Encoding Business-Value Ranking into Document Identifiers for Generative Retrieval (Ling et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.38.pdf