Sampling-Free Context Distillation for Masked Diffusion Large Language Models

Diffusion large language models (dLLMs) generate responses by iteratively unmasking tokens in parallel, yet this structural property has been exploited primarily for inference acceleration rather than for training. In this work, we show that two properties of masked diffusion models can be combined into an efficient and principled supervision signal. First, parallel masked-token prediction yields logit distributions over all masked positions in a single forward pass, enabling dense token-level supervision without sequential rollouts. Second, the bidirectional attention of dLLMs allows a teacher conditioned on the ground-truth answer to produce a privileged distribution over all remaining masked positions, also extractable in a single forward pass. Building on these observations, we propose Sampling-Free Context Distillation (SFCD), which trains the student-conditioned only on the question-to match the answer-conditioned teacher distributions at every masked position. This eliminates the need for iterative denoising rollouts, bypassing the high variance and computational overhead of rollout-based post-training methods such as GRPO. We evaluate SFCD on four scientific reasoning domains from SciKnowEval, as well as mathematics and code generation benchmarks. SFCD substantially outperforms GRPO-based training on scientific reasoning tasks while achieving 8 -13x faster convergence, and matches rollout-based methods on mathematics and coding at reduced training cost. We further analyze the effects of teacher parameterization, divergence objectives, and parallel decoding efficiency through ablations.