Large Language Models (LLMs) excel at text generation, but still struggle to produce accurate, long-form documents grounded in large knowledge bases. Existing “top-down” approaches—which plan before retrieving evidence—often result in fragmented or unfaithful content due to misalignment between the plan and available knowledge.

To overcome this, we introduce a “bottom-up” framework based on a “Retrieval-First for Knowledge, Clustering for Structure” strategy. Our method begins by establishing the “knowledge boundaries” of the source corpus through comprehensive retrieval, followed by unsupervised clustering to group evidence into distinct “knowledge clusters.” These clusters objectively inform a hierarchical outline and subsequent text generation, ensuring strict adherence to the source and significantly reducing hallucination.

Experimental results on both 14B- and 32B-parameter models demonstrate that our method achieves performance comparable to or exceeding state-of-the-art baselines, and is expected to demonstrate unique advantages in knowledge-constrained scenarios that demand high fidelity and structural coherence. Our work presents an effective paradigm for generating reliable, structured, long-form documents, paving the way for more robust LLM applications in high-stakes, knowledge-intensive domains.

Retrieval-Augmented Generation (RAG) has substantially advanced knowledge-intensive question answering (QA). Existing methods typically follow a ’decompose and sequentially solve’ paradigm, mainly optimizing sub-question generation, retrieval, or search decisions, while lacking explicit maintenance of cross-step intermediate reasoning states. As a result, retrieval noise, query drift, and local reasoning errors introduced in early steps can be inherited and amplified along the reasoning chain, eventually causing reasoning failure. We propose **CloRe**, a state-based closed-loop reasoning framework for multi-hop question answering that explicitly maintains hop-level intermediate states and coordinates forward review with dependency-aware backward correction. In the forward phase, CloRe uses conditional query rewriting to mitigate query drift, stage-wise evidence acquisition to mitigate retrieval noise, and step-level review with local repair to filter unreliable intermediate answers before they are propagated. When forward correction is insufficient, CloRe enters a backward phase that uses downstream failure signals to backtrack along explicit dependencies and revise predecessor states that were locally acceptable under earlier review but become incompatible with later dependency constraints. Experiments on three representative multi-hop QA benchmarks show that CloRe achieves competitive performance against strong baselines, demonstrating the effectiveness of explicit intermediate-state maintenance for multi-hop reasoning.

Knowledge-intensive multi-hop question answering (QA) tasks, which require integrating evidence from multiple sources to address complex queries, often necessitate multiple rounds of retrieval and iterative generation by large language models (LLMs). However, incorporating many documents and extended contexts poses challenges—such as hallucinations and semantic drift—for lightweight LLMs with fewer parameters. This work proposes a novel framework called DEC (Dynamic Enhancement Chain). DEC first decomposes complex questions into logically coherent subquestions to form a hallucination-free reasoning chain. It then iteratively refines these subquestions through context-aware rewriting to generate effective query formulations. For retrieval, we introduce a lightweight discriminative keyword extraction module that leverages extracted keywords to achieve targeted, precise document recall with relatively low computational overhead. Extensive experiments on three multi-hop QA datasets demonstrate that DEC performs on par with or surpasses state-of-the-art benchmarks while significantly reducing token consumption. Notably, our approach attains state-of-the-art results on models with 8B parameters, showcasing its effectiveness in various scenarios, particularly in resource-constrained environments.

Tool invocation significantly enhances the capabilities of Large Language Models (LLMs), yet challenges persist, particularly in complex task scenarios. Current methods, such as instruction-enhanced reasoning and supervised fine-tuning, often result in unnecessarily long reasoning paths and face difficulties in verifying the correctness of intermediate steps. In this paper, we propose CodeTool, a novel framework for stepwise code generation that improves LLM tool invocation by leveraging the concise and easily verifiable nature of code. CodeTool incorporates two distinct process rewards: the On-the-spot Reward, which provides immediate feedback on the accuracy of each tool invocation, and the Latent Reward, which assesses the contribution of each step toward overall task completion. By maximizing the cumulative reward of the On-the-spot and Latend Rewards at each step, LLMs are guided to follow efficient and accurate reasoning paths. Extensive experiments on StableToolBench and RestBench-TMDB demonstrate the superiority of CodeTool over existing approaches.