Ziheng Chen
Author directoryPapers on this page may belong to the following people: Ziheng Chen, Ziheng Chen
2026
SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion
Chengting Yu | Xiaobo Shu | Yadao Wang | Yizhen Zhang | Haoyi Wu | You Wu | Rujiao Long | Ziheng Chen | Yuchi Xu | Wenbo Su | Bo Zheng
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chengting Yu | Xiaobo Shu | Yadao Wang | Yizhen Zhang | Haoyi Wu | You Wu | Rujiao Long | Ziheng Chen | Yuchi Xu | Wenbo Su | Bo Zheng
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Recursive (looped) Transformers decouple computational depth from parameter depth by repeatedly applying shared layers, providing an explicit architectural primitive for iterative refinement and latent reasoning. However, early looped Transformers often underperform non-recursive baselines of equal compute. While recent literature has introduced more effective recursion mechanisms to mitigate this gap, existing architectures still operate at a fixed, full-token resolution, neglecting the potential efficiency of computing over compressed latent representations. In this paper, we propose SpiralFormer, a looped Transformer that executes recurrence under a multi-resolution recursion schedule. We provide probing evidence that multi-resolution recursion enables the model to learn hierarchical dependencies by inducing iteration-wise functional specialization across different scales. Empirically, SpiralFormer achieves better parameter and compute efficiency than both looped and non-looped baselines across model scales from 160M to 1.4B, establishing sequence resolution as a potential axis for scaling recursive architectures.
2022
AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension
Xiao Li | Gong Cheng | Ziheng Chen | Yawei Sun | Yuzhong Qu
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Xiao Li | Gong Cheng | Ziheng Chen | Yawei Sun | Yuzhong Qu
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent machine reading comprehension datasets such as ReClor and LogiQA require performing logical reasoning over text. Conventional neural models are insufficient for logical reasoning, while symbolic reasoners cannot directly apply to text. To meet the challenge, we present a neural-symbolic approach which, to predict an answer, passes messages over a graph representing logical relations between text units. It incorporates an adaptive logic graph network (AdaLoGN) which adaptively infers logical relations to extend the graph and, essentially, realizes mutual and iterative reinforcement between neural and symbolic reasoning. We also implement a novel subgraph-to-node message passing mechanism to enhance context-option interaction for answering multiple-choice questions. Our approach shows promising results on ReClor and LogiQA.