Kaitai Ou-yang
Author directory2026
PaperGraph-Bench: Graph-Guided Benchmark Construction and Multi-Turn Evaluation for Scientific Paper Reading
Rao Fu | Qing Dai | Kaitai Ou-yang | Kang He | Fei Li | Donghong Ji | Chong Teng
Findings of the Association for Computational Linguistics: EMNLP 2026
Rao Fu | Qing Dai | Kaitai Ou-yang | Kang He | Fei Li | Donghong Ji | Chong Teng
Findings of the Association for Computational Linguistics: EMNLP 2026
Scientific-paper reading requires models to recover a paper’s core argument, connect claims to textual and multimodal evidence, and avoid unsupported inferences across dialogue turns. We introduce PaperGraph-Bench, a graph-guided benchmark that turns each paper into a paper-specific evaluation state graph with macro stages, evidence-grounded knowledge components (KCs), verified reasoning edges, multimodal assets, rubrics, and trajectory state. The graph drives macro coverage questions, challenge questions, reasoning threads, and bounded repair turns, while the evaluator tracks KC coverage, challenge failures, hallucination events, and repair outcomes. Experiments on a human-audited prototype with ten models show that coverage, robustness, hallucination type, and repair behavior reveal distinct paper-reading abilities. Our code and current dataset are available at https://github.com/furao0011/PaperGraphBench.