Before You Scale: The Cost of Supervision Mismatch in PRM Distillation

Process reward models (PRMs) are often trained from automatically generated step-level supervision. However, when a small verifier trained in this way fails, it is often unclear whether the cause is student capacity, the training recipe, or the supervision itself. This distinction has direct practical consequences: when supervision itself is the bottleneck, any compute spent scaling the student or tuning the recipe is wasted until the data problem is fixed. We hit this ambiguity in practice. Privileged (answer-aware) labeling improves our teacher on MATH (+0.051 first-error F1), yet a 1.5B student distilled from those labels is statistically indistinguishable from one trained on answer-blind labels, even though a positive control shows the pipeline does track coarse teacher quality. To isolate the cause, we fix the student and evaluation path, varying only the supervision. Gold labels from non-MATH splits transfer reliably to ProcessBench-MATH (GSM8K-source 0.7515 mean ROC-AUC over four seeds; OmniMath-source 0.7694), while OlympiadBench-source supervision is high-variance (0.585–0.716), showing format match alone is not sufficient. Generated teacher labels are markedly weaker under the same path (four-seed no-GT rank-loss baseline: 0.6062). The failure is primarily one of rendering convention: re-rendering the same generated labels into the ProcessBench format raises the matched same-source baseline from 0.6183 to 0.7366. The practical conclusion for efficient PRM development is that a cheap source- and format-matched diagnostic, roughly 0.3 GPU-hours against the ~15 GPU-hours of the recipe sweep it replaces, can reveal supervision mismatch before further compute is committed to larger students or broader recipe searches.