How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding?We study this question through Speculative Refinement (SpecRef), a training-free hybrid method that warm-starts a masked diffusion language model from an AR draft using entropy-guided selective masking.Evaluating SpecRef across six benchmarks (HumanEval, MBPP, GSM8K, BBH, ARC-Challenge, HellaSwag) with three distinct evaluation protocols (execution-based pass@1, exact-match, log-likelihood scoring), we surface several findings relevant beyond our specific system:(1) code benchmarks conflate structural discovery with logical correctness: providing a syntactic scaffold lifts accuracy from near zero to over 20% without changing the model, indicating that much of the baseline failure is structural;(2) a refinement tension phenomenon where multi-stage correction degrades already-correct tokens, exposing benchmark saturation ceilings invisible to single-model evaluation;(3) log-likelihood and generative evaluation produce different model rankings for the same model pair, suggesting they measure different capabilities;(4) standard Python post-processing silently breaks code evaluation for non-AR generators.These observations apply to any multi-stage or non-autoregressive generation pipeline and point toward more diagnostic evaluation practices.

Masked diffusion language models reuse one denoiser across many steps, so post-training quantization (PTQ) must balance predictive quality, serialized storage, and runtime memory. We evaluate LLaDA-8B-Instruct and run transfer checks on Dream-7B, MDLM-OWT, encoder-only masked LMs, LLaDA-MoE, and LLaDA2.0-mini. The 24-text calibration set is distinct from the main n=500-example-per-task evaluation. Three findings emerge. First, AWQ-4 is the closest tested 4-bit quality point at the evaluated 1.1–2.3 percentage-point resolution, although ARC-Challenge shows a detectable loss. TurboLDM-(3+1) instead provides the smallest serialized checkpoint (4.82 GiB, 3.55 effective bits) while losing quality, making it a storage-compression and diagnostic result, not a live-memory result. NF4 alone reduces peak VRAM from roughly 15 GiB to 6 GiB in our tested PyTorch/bitsandbytes runtime; this is a runtime-specific outcome, not an inherent format advantage. Second, individual attention projections are inexpensive to quantize, whereas jointly quantizing the attention block is damaging. Third, more denoising steps partly recover GSM8K loss but not multiple-choice accuracy.