SciCompanion: Graph-Grounded Reasoning for Structured Evaluation of Scientific Arguments

Joshua Alan Flashner, Adithya Kulkarni, Dawei Zhou


Abstract
The exponential growth of scientific publications has overwhelmed reviewers and researchers, with top conferences receiving thousands of submissions annually. Reviewers must assess feasibility, novelty, and impact under tight deadlines, often lacking tools to identify relevant prior work. Early-career researchers face similar challenges, with limited support to navigate fast-evolving fields. Existing LLM-based systems struggle with static retrieval, surface-level features, and lack multi-hop reasoning, leading to shallow or hallucinated assessments. Scientific evaluation requires a deep, relational understanding, which current retrieval-augmented generation (RAG) methods fail to achieve. We introduce SciCompanion, a graph-grounded reasoning framework for structured scientific evaluation. Given a paper or abstract-like input, SciCompanion builds a dynamic knowledge graph from recent publications, domain-specific databases, and curated metadata. It employs multi-hop reasoning to iteratively construct contextual graphs and generate structured critiques, enabling deeper exploration of scientific literature. Unlike sentiment-biased LLM evaluations, SciCompanion directly optimizes retrieval and graph refinement using Group Relative Policy Optimization (GRPO), producing reviews aligned with expert judgments. Experiments on ICLR and ACL datasets show that SciCompanion reduces evaluation error by over 30% compared to prompting-only baselines and allows smaller models to outperform larger ones. Evaluations across three datasets, using metrics for retrieval accuracy, semantic overlap, and multi-hop sensitivity, along with a case study, demonstrate SciCompanion’s robustness and versatility.
Anthology ID:
2025.findings-emnlp.1315
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
24223–24244
Language:
URL:
https://aclanthology.org/2025.findings-emnlp.1315/
DOI:
Bibkey:
Cite (ACL):
Joshua Alan Flashner, Adithya Kulkarni, and Dawei Zhou. 2025. SciCompanion: Graph-Grounded Reasoning for Structured Evaluation of Scientific Arguments. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 24223–24244, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
SciCompanion: Graph-Grounded Reasoning for Structured Evaluation of Scientific Arguments (Flashner et al., Findings 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.findings-emnlp.1315.pdf
Checklist:
 2025.findings-emnlp.1315.checklist.pdf