Jinghan Wu
Author directory2026
Succinctness, Not Expressivity, Predicts Syntactic Competence
Shangao Li | Jinghan Wu | Chang Lu | Yizheng Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Shangao Li | Jinghan Wu | Chang Lu | Yizheng Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Transformers outperform state-space models and recurrent architectures on long-distance syntactic dependencies, yet no theory predicts which architecture will win on which task. For syntactic phenomena expressible as short B-RASP programs, Transformers need a fixed number of layers regardless of dependency distance, while sequential models face growing effective complexity. We derive a *succinctness prediction table* ranking Transformers, LSTMs, and Mamba on six English syntactic phenomena before any experiment, with GRU and RWKV as additional sequential controls. All predictions are confirmed: Transformer accuracy exceeds 96% at maximum structural complexity while Mamba drops below 55%, a gap that persists at 79\\times the Transformer’s parameter count. To verify the winning architecture succeeds *for the predicted reason*, Distributed Alignment Search (DAS), a causal intervention method, finds alignment of 0.95+ in Transformers but near the 0.50 chance baseline in Mamba, validated by seven families of negative controls. Training dynamics reveal that Transformers assemble the predicted algorithm in causal-dependency order, whereas Mamba achieves high accuracy without it, producing an *accuracy-alignment gap* invisible to behavioral benchmarks. Validation on Pythia confirms these patterns transfer to pre-trained models.