Retrieval-Augmented Generation (RAG) effectively mitigates hallucinations and knowledge obsolescence in Large Language Models (LLMs). However, prevailing RAG systems typically retrieve evidence from discrete chunks or through decoupled structures, making it difficult to jointly preserve cross-document navigability and within-document coherence on complex tasks. To address this challenge, we propose TSA-RAG (Tiered-Structure-Aware RAG), a unified framework that recasts retrieval as structure-preserving navigation over a Tiered Document Graph (TDG). The TDG offers a joint retrieval space that explicitly couples relational topology with hierarchical skeletons, enabling seamless traversal across granularities. Building on this representation, TSA-RAG navigates evidence through a progressive Expand-Route-Retrieve paradigm, which expands query-focused subgraphs under path-consistency constraints, routes the query to structural entry points within relevant documents, and retrieves subtree-structured evidence preserving discourse continuity. Extensive experiments on seven QA datasets demonstrate that TSA-RAG consistently outperforms strong baselines while yielding more compact retrieval contexts. Quantitative analyses further confirm the scalability and robustness of our framework. Code is available at https://github.com/ZKYQUQ/TSA-RAG.

While large language models (LLMs) demonstrate impressive capabilities, their reliance on parametric knowledge often leads to factual inaccuracies. Retrieval-Augmented Generation (RAG) mitigates this by leveraging external documents, yet existing approaches treat retrieved passages as isolated chunks, ignoring valuable structure that is crucial for document organization. Motivated by this gap, we propose Retrieve-DocumentRoute-Read (RDR2), a novel framework that explicitly incorporates structural information throughout the RAG process. RDR2 employs an LLM-based router to dynamically navigate document structure trees, jointly evaluating content relevance and hierarchical relationships to assemble optimal evidence. Our key innovation lies in formulating document routing as a trainable task, with automatic action curation and structure-aware passage selection inspired by human reading strategies. Through comprehensive evaluation on five challenging datasets, RDR2 achieves state-of-the-art performance, demonstrating that explicit structural awareness significantly enhances RAG systems’ ability to acquire and utilize knowledge, particularly in complex scenarios requiring multi-document synthesis.