Reasoning with Trees: Faithful Question Answering over Knowledge Graph

Tiesunlong Shen, Jin Wang, Xuejie Zhang, Erik Cambria


Abstract
Recent advancements in large language models (LLMs) have shown remarkable progress in reasoning capabilities, yet they still face challenges in complex, multi-step reasoning tasks. This study introduces Reasoning with Trees (RwT), a novel framework that synergistically integrates LLMs with knowledge graphs (KGs) to enhance reasoning performance and interpretability. RwT reformulates knowledge graph question answering (KGQA) as a discrete decision-making problem, leveraging Monte Carlo Tree Search (MCTS) to iteratively refine reasoning paths. This approach mirrors human-like reasoning by dynamically integrating the LLM’s internal knowledge with external KG information. We propose a real-data guided iteration technique to train an evaluation model that assesses action values, improving the efficiency of the MCTS process. Experimental results on two benchmark KGQA datasets demonstrate that RwT significantly outperforms existing state-of-the-art methods, with an average performance improvement of 9.81%. Notably, RwT achieves these improvements without requiring complete retraining of the LLM, offering a more efficient and adaptable approach to enhancing LLM reasoning capabilities.
Anthology ID:
2025.coling-main.211
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3138–3157
Language:
URL:
https://aclanthology.org/2025.coling-main.211/
DOI:
Bibkey:
Cite (ACL):
Tiesunlong Shen, Jin Wang, Xuejie Zhang, and Erik Cambria. 2025. Reasoning with Trees: Faithful Question Answering over Knowledge Graph. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3138–3157, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
Reasoning with Trees: Faithful Question Answering over Knowledge Graph (Shen et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.211.pdf