Generative Retrieval (GR) aims to directly generate document identifiers (DocIDs) without external indices. We identify granularity collapse: during training, GR models develop a preference for generating DocIDs with a specific granularity, causing retrieval failures when it differs from that of the target document’s DocID. We propose to quantify this mismatch using "isotropy" and confirm that retrieval accuracy degrades as such granularity mismatch increases. To address this, we propose Granularity Routing for Generative Retrieval (GR²), which trains multiple expert models specialized for different granularity levels and routes each query to the appropriate expert. Experiments on NQ320k demonstrate that improves retrieval performance, with larger gains on queries that suffer more from granularity collapse.

Generative Retrieval (GR) maps queries to documents by generating discrete identifiers (DocIDs).However, offline DocID assignment and constrained decoding often prevent GR from capturing query-specific intent, especially when documents express multiple or unseen intents (i.e., intent misalignment).We introduce Dynamic Docid Decoding (D3), an inference-time mechanism that adaptively refines DocIDs through delayed, query-informed identifier expansion.D3 uses (a) verification to detect intent misalignment and (b) dynamic decoding to extend DocIDs with query-aligned tokens, even those absent from the pre-indexed vocabulary, enabling plug-and-play DocID expansion beyond the static vocabulary while adding minimal overhead.Experiments on NQ320k and MS-MARCO show that D3 consistently improves retrieval accuracy, especially on unseen and multi-intent documents, across various GR models, including a +2.4%p nDCG@10 gain on the state-of-the-art model.