Zhaolong Su
Author directory2026
Triplet-Block Diffusion RWKV
Ke Lin | Yiyang Luo | Zhaolong Su | Yunya Song | Anyi Rao
Findings of the Association for Computational Linguistics: EMNLP 2026
Ke Lin | Yiyang Luo | Zhaolong Su | Yunya Song | Anyi Rao
Findings of the Association for Computational Linguistics: EMNLP 2026
Causal Transformer language models suffer from strictly sequential decoding and a quadratic per-step attention cost, resulting in expensive inference and an inability to revise previous tokens during text generation. While linear-time causal models and discrete diffusion models each address these weaknesses, their integration remains inherently inconsistent and underdeveloped. In this work, we propose B3D-RWKV, a diffusion-based RWKV variant that integrates the model’s 𝒪(L) inference efficiency with parallel, bidirectional discrete-diffusion through a triplet-block layout method. B3D-RWKV-7.2B reaches comparable accuracy on an 8-task suite versus existing models while significantly outperforming baselines in decoding throughput with an average of 1.6× speedup. Code is available at https://github.com/leonardodalinky/B3D-RWKV.