Retrieval-Augmented Generation (RAG) was proposed to address the hallucination question of large language models (LLMs). However, the traditional RAG framework has certain limitations: for simple questions, the search results often introduce a large amount of irrelevant information; while for complex questions, the lengthy reference knowledge provided by the retrieval lacks structural information. Therefore, we proposed a structure-aware RAG, which achieves noise removal in retrieval through multi-chain graph navigation reasoning(Trig-Nav). This method constructs question triple reasoning chains and reference knowledge graphs with text attributes, allowing the system to retrieve three types of knowledge along different paths based on the requirements of LLM. It provides LLM with multi-angle and structured information input and significantly reduces noise. We conducted a comprehensive evaluation of Trig-Nav, comparing it with baseline methods across multiple datasets.Compared to traditional RAG, there is an average improvement of 6% in effectiveness. The results showed that Trig-Nav significantly enhances the model’s performance, validating the effectiveness of this approach.

Large language models (LLMs) show strong reasoning and decision-making ability, but their high inference cost motivates transferring agentic skills to small language models (SLMs). Agent distillation trains SLMs on full reason–act–observe trajectories from a tool-using teacher, enabling SLMs to acquire the tool-use capabilities of large teacher models. However, some teacher-agent trajectories are simply hard for the student to learn, and their compatibility with the student can vary widely; moreover, a uniform token-level loss prevents SLMs from learning the tool-use patterns and final decisions that truly drive successful reasoning. Therefore, we propose SmartAD, a capacity-aligned agent distillation framework that improves both the distilled data and the supervision signal. SmartAD (i) selects, for each training example, the trajectory with the minimum negative log-likelihood among multiple correct teacher samples to obtain student-friendly training data, and (ii) applies a segment-weighted loss that emphasizes action execution and final decision spans over intermediate reasoning. Experiments on multi-hop QA and math benchmarks with 1.5B and 3B models show that SmartAD consistently outperforms all baselines. Overall, our method enables small models to learn the teacher’s capabilities more easily and efficiently through trajectory selection and segment-weighted supervision, achieving capacity-aligned distillation.

Graph learning requires jointly leveraging node semantics and graph structure, but traditional methods often suffer from label scarcity and shallow semantic utilization. Although LLMs provide strong semantic reasoning for graph learning, existing LLM-augmented pipelines face two major challenges: their semantics-only outputs can inject semantic noise that conflicts with graph topology, and they often incur substantial overhead due to LLM fine-tuning or repeated LLM calls at training/inference. We propose CoFGraph, a confidence-feedback framework that interleaves reciprocal calibration between semantic and structural views to enable reliable and efficient LLM-augmented graph learning. To avoid expensive LLM adaptation, we keep the LLM frozen and train a lightweight Interpreter as a semantic proxy, eliminating the need for LLM fine-tuning and continual test-time prompting. To further mitigate semantic noise and the lack of structural feedback, we introduce confidence-based dual-view learning with node-wise confidence estimation and dynamic role switching, allowing the more reliable view to correct the other. Experiments on four public benchmarks validate the effectiveness of CoFGraph and demonstrate strong stability across varying training label ratios, with particularly consistent gains in low-resource regimes.

Recent advancements in Large Language Models (LLMs) are increasingly driven by using LLMs themselves to synthesize large amounts of instruction-tuning data. To transcend the limitations of single model synthesis, recent research has shifted toward multiple model data synthesis, where distinct models assume specialized roles to synthesize data through interaction. However, existing feedback cannot distinguish truly high-quality instructions, those that promote complex reasoning, deviate from the learned distribution, and require multi-source knowledge, due to coarse granularity and signal dilution. In human cognition, truly high-quality instructions require greater mental effort—the **Cognitive Load**—to address. In this paper, we propose a paradigm shift in feedback. Instead of external, result-oriented evaluation, we propose an internal, process-aware feedback based on target models’ computational effort. The novel feedback, namely **Inference Stress Index (ISI)**, quantifies instruction quality in a fine-grained, dilution-resistant manner by focusing on high-uncertainty tokens. Building on this, we present the **Co**gnitive-**F**eedback **A**daptive **G**eneration (**CoFAG**) framework, a self-evolving multiple model collaboration paradigm involving a synthesizer model, a fixer model, and a collaborator model. Extensive experiments demonstrate that CoFAG significantly outperforms other data synthesis techniques on complex reasoning benchmarks, validating the superiority of using internal cognitive feedback for generating high-quality data.

Knowledge graphs are dynamic structures that continuously evolve as new entities emerge, often accompanied by only a handful of associated triples. Current knowledge graph reasoning methods struggle in these few-shot scenarios due to their reliance on extensive structural information.To address this limitation, we introduce ENGRAM, a novel approach that enables inductive reasoning on few-shot KGs by innovatively enriching the semantics from both textual and structural perspectives. Our key innovation lies in designing a task-aware language model that activates the language model’s in-context learning ability for structured KG tasks, effectively bridging the gap between unstructured natural language and structured tasks. Unlike prior methods that inefficiently employ classification over exhaustive candidate sets, we recast knowledge graph reasoning from a generative perspective, allowing for direct computation of inference results without iterative enumeration. Additionally, we propose a distant neighborhood awareness strategy to enrich the sparse structural features of few-shot entities.Our experimental findings indicate that our method not only achieves state-of-the-art performance in few-shot scenarios. The tunable parameters of our model are approximately 1% of those in previous language model-based methods, and the inference time has been reduced to 1/10 of that required by previous methods.

Complex multi-hop questions often require comprehensive retrieval and reasoning. As a result, effectively parsing such questions and establishing an efficient interaction channel between large language models (LLMs) and knowledge graphs (KGs) is essential for ensuring reliable reasoning. In this paper, we present a novel semantic parsing framework Correcting on Graph (CoG), aiming to establish faithful logical queries that connect LLMs and KGs. We first propose a structured knowledge decoding that enables the LLM to generate fact-aware logical queries during inference, while leveraging its parametric knowledge to fill in the blank intermediate entities. Then, we introduce a knowledge path correction that combines the logical query with KGs to correct hallucination entities and path deficiencies in the generated content, ensuring the reliability and comprehensiveness of the retrieved knowledge. Extensive experiments demonstrate that CoG outperforms the state-of-the-art KGQA methods on two knowledge-intensive question answering benchmarks. CoG achieves a high answer hit rate and exhibits competitive F1 performance for complex multi-hop questions.

Temporal evolution attribute graph prediction, a key task in graph machine learning, aims to forecast the dynamic evolution of node attributes over time. While recent advances in Large Language Models (LLMs) have enabled their use in enhancing node representations for integration with Graph Neural Networks (GNNs), their potential to directly perform GNN-like aggregation and interaction remains underexplored. Furthermore, traditional approaches to initializing attribute embeddings often disregard structural semantics, limiting the provision of rich prior knowledge to GNNs. Current methods also primarily focus on 1-hop neighborhood aggregation, lacking the capability to capture complex structural interactions. To address these limitations, we propose a novel prediction framework that integrates structural information into attribute embeddings through the introduction of an attribute embedding loss. We design specialized prompts to enable LLMs to perform GNN-like aggregation and incorporate a relation-aware Graph Convolutional Network to effectively capture long-range and complex structural dependencies. Extensive experiments on multiple real-world datasets validate the effectiveness of our approach, demonstrating significant improvements in predictive performance over existing methods.

Large Language Models (LLMs) have demonstrated a remarkable understanding of language nuances through instruction tuning, enabling them to effectively tackle various natural language processing tasks. Recent research has focused on the quality of instruction data rather than the quantity of instructions. However, existing high-quality instruction selection methods rely on external models or rules, overlooking the intrinsic association between pre-trained model and instruction data, making it difficult to select data that align with the preferences of pre-trained model. To address this challenge, we propose a strategy that utilizes noise injection to identify the quality of instruction data, without relying on external model. We also implement the strategy of combining inter-class diversity and intra-class diversity to improve model performance. The experimental results demonstrate that our method significantly outperforms the model trained on the entire dataset and established baselines. Our study provides a new perspective on noise injection in the field of instruction tuning, and also illustrates that the pre-trained model itself should be considered in defining high-quality. Additionally, we publish our selected high-quality instruction data.