Reasoning-intensive retrieval often requires evidence whose relevance is not captured by keyword or embedding similarity alone, exposing a core limitation of traditional IR systems. An emerging direction known as LLM-guided hierarchical search addresses this limitation by letting an LLM directly traverse a semantic tree over the corpus. Such methods so far work well only with a frontier LLM in the search loop, which makes inference expensive. We propose HSearch-R1, a distillation recipe for learning compact (4B) LLMs that perform efficient LLM-guided hierarchical search. The recipe distils a frontier teacher’s per-step value estimates from its search trajectories into a much smaller LLM (Qwen3.5-4B) via RL, using a composite reward that matches the teacher’s ranking and score distribution at each step; this outcome-level RL distillation substantially outperforms supervised fine-tuning on teacher outputs. We further propose a retrieval-prior-informed variant of the search that injects a cheap retriever’s shortlist as a prior into the tree and adds robustness when the offline index is stale. The base HSearch-R1-4B matches or closely tracks frontier LLMs on the seven static BRIGHT-StackExchange subsets at roughly 50× lower per-query cost. Adding the retrieval prior brings HSearch-R1 to 52.5 nDCG@10 on StackExchange and 45.6 across all 12 BRIGHT subsets, within 1 nDCG of the strongest published ensembled baselines while remaining substantially cheaper.

Extreme Multi-label Text Classification (XMC) entails selecting the most relevant labels for an instance from a vast label set. Extreme Zero-shot XMC (EZ-XMC) extends this challenge by operating without annotated data, relying only on raw text instances and a predefined label set, making it particularly critical for addressing cold-start problems in large-scale recommendation and categorization systems. State-of-the-art methods, such as MACLR and RTS, leverage lightweight bi-encoders but rely on suboptimal pseudo labels for training, such as document titles (MACLR) or document segments (RTS), which may not align well with the intended tagging or categorization tasks. On the other hand, LLM-based approaches, like ICXML, achieve better label-instance alignment but are computationally expensive and impractical for real-world EZ-XMC applications due to their heavy inference costs. In this paper, we introduce LMTX (Large language Model as Teacher for eXtreme classification), a novel framework that bridges the gap between these two approaches. LMTX utilizes an LLM to identify high-quality pseudo labels during training, while employing a lightweight bi-encoder for efficient inference. This design eliminates the need for LLMs at inference time, offering the benefits of improved label alignment without sacrificing computational efficiency. Our approach achieves superior performance and efficiency over both LLM and non-LLM based approaches, establishing a new state-of-the-art in EZ-XMC.