Reasoning-intensive retrieval often requires evidence whose relevance is not captured by keyword or embedding similarity alone, exposing a core limitation of traditional IR systems. An emerging direction known as LLM-guided hierarchical search addresses this limitation by letting an LLM directly traverse a semantic tree over the corpus. Such methods so far work well only with a frontier LLM in the search loop, which makes inference expensive. We propose HSearch-R1, a distillation recipe for learning compact (4B) LLMs that perform efficient LLM-guided hierarchical search. The recipe distils a frontier teacher’s per-step value estimates from its search trajectories into a much smaller LLM (Qwen3.5-4B) via RL, using a composite reward that matches the teacher’s ranking and score distribution at each step; this outcome-level RL distillation substantially outperforms supervised fine-tuning on teacher outputs. We further propose a retrieval-prior-informed variant of the search that injects a cheap retriever’s shortlist as a prior into the tree and adds robustness when the offline index is stale. The base HSearch-R1-4B matches or closely tracks frontier LLMs on the seven static BRIGHT-StackExchange subsets at roughly 50× lower per-query cost. Adding the retrieval prior brings HSearch-R1 to 52.5 nDCG@10 on StackExchange and 45.6 across all 12 BRIGHT subsets, within 1 nDCG of the strongest published ensembled baselines while remaining substantially cheaper.

Despite rapid progress in large language models (LLMs), their performance on a vast majority of languages remains unsatisfactory. In this paper, we study building language-specific LLMs by adapting monolingual and multilingual LLMs. We conduct systematic experiments on how design choices (base model selection, vocabulary extension, and continued pretraining) impact the adapted LLM, both in terms of efficiency (how many tokens are needed to encode the same amount of information) and end task performance. We find that (1) the initial performance of LLM does not always correlate with the final performance after the adaptation. Adapting an English-centric models can yield better results than adapting multilingual models despite their worse initial performance on low-resource languages. (2) Efficiency can easily improved with simple vocabulary extension and continued pretraining in most LLMs we study, and (3) The optimal adaptation method (choice of the base model, new vocabulary size, training data, initialization strategy) is highly language-dependent, and the simplest embedding initialization works well across various experimental settings. Together, our work lays foundations on efficiently building language-specific LLMs by adapting existing LLMs.