Liang Zeng
Author directory2026
From Tokens to Topology: When Large Language Models Meet Graph-Structured Reasoning
Qihang Zhang | Liang Zeng
Findings of the Association for Computational Linguistics: EMNLP 2026
Qihang Zhang | Liang Zeng
Findings of the Association for Computational Linguistics: EMNLP 2026
Reasoning benchmarks often collapse heterogeneous problem types into aggregate scores, obscuring which structural computations large language models (LLMs) can reliably perform. Existing graph-reasoning benchmarks cover graph traversal, broad graph computation, and graph-language interaction, but do not directly test whether graph-structured reasoning transfers across topological mechanisms under matched conditions. We introduce GraphReason-Bench (GR-Bench), comprising 1,800 programmatically generated and oracle-verified problems across three prompt-length regimes and six task families: local motif enumeration, exact-depth breadth-first search, bounded path enumeration, articulation-point identification, non-induced subgraph matching, and non-monotonic rule execution. All tasks share a text-only executable interface, exact oracles, canonical output normalization, and set-based scoring, enabling controlled cross-mechanism comparison. On the 600-instance main evaluation split across nine proprietary and open-source LLMs, we find mechanism-level dissociation: models that succeed on one structural operation often fail on others, and cross-task performance does not reduce to a stable graph-reasoning ability. Trace-level cases reveal mechanism-specific failures such as premature search termination and loss of intermediate state. An initial sparse-autoencoder probe, supported by cross-model robustness checks, shows that topology-sensitive representations become less separable as structural difficulty increases. Together, these results establish graph-structured reasoning as a measurable domain of LLM evaluation.