Niraj Jha
Author directory2026
CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models
Yihao Liang | Ze Wang | Ximeng Sun | Jiang Liu | Jialian Wu | Hao Chen | Xiaodong Yu | Emad Barsoum | Zicheng Liu | Niraj Jha
Findings of the Association for Computational Linguistics: EMNLP 2026
Yihao Liang | Ze Wang | Ximeng Sun | Jiang Liu | Jialian Wu | Hao Chen | Xiaodong Yu | Emad Barsoum | Zicheng Liu | Niraj Jha
Findings of the Association for Computational Linguistics: EMNLP 2026
Diffusion language models (DLMs) accelerate generation by decoding multiple tokens in parallel, but pushing them to fast decoding often sacrifices accuracy. We attribute this to a static-to-dynamic mismatch: training optimizes local denoising under fixed schedules, while fast inference skips steps and encounters irregular mask patterns. Existing accelerators largely address only one side of this problem. Training-free inference methods speed up decoding without changing the model, leaving the mismatch unresolved; training-side distillation methods adapt the model, but often tie the student to a specific decoding trajectory, limiting composability and still suffering accuracy loss. We propose CD4LM, a composable framework for accurate DLM acceleration. Its core training component, Discrete-Space Consistency Distillation (DSCD) trains a trajectory-invariant student on diverse masked states, reducing the mismatch without fixing a decoding policy. We pair DSCD with a lightweight Confidence-Adaptive Decoder, while keeping the trained student compatible with external inference-side accelerators. On GSM8K, CD4LM preserves accuracy while achieving a 5.18× speedup; stacking CD4LM with per-step cache accelerators further pushes speedup to 9.73× at teacher-level accuracy, suggesting a scalable path for combining training-side robustness with future decoding advances.
2024
DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling
Shikhar Tuli | Chi-Heng Lin | Yen-Chang Hsu | Niraj Jha | Yilin Shen | Hongxia Jin
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Shikhar Tuli | Chi-Heng Lin | Yen-Chang Hsu | Niraj Jha | Yilin Shen | Hongxia Jin
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Traditional language models operate autoregressively, i.e., they predict one token at a time. Rapid explosion in model sizes has resulted in high inference times. In this work, we propose DynaMo, a suite of multi-token prediction language models that reduce net inference times. Our models dynamically predict multiple tokens based on their confidence in the predicted joint probability distribution. We propose a lightweighttechnique to train these models, leveraging the weights of traditional autoregressive counterparts. Moreover, we propose novel ways to enhance the estimated joint probability to improve text generation quality, namely co-occurrence weighted masking and adaptive thresholding. We also propose systematic qualitative and quantitative methods to rigorously test the quality of generated text for non-autoregressive generation. One of the models in our suite, DynaMo-7.3B-T3, achieves same-quality generated text as the baseline (Pythia-6.9B) while achieving 2.57× speed-up with only 5.87% and 2.67% parameter and training time overheads, respectively.
Search