When the Generator Grades Itself: A Self-Judge Under-Flags Unsupported Outputs at Matched Scale in LLM Social Simulation

A researcher who validates an LLM social simulation with an automated fidelity check rarely sees which model ran that check. The cheapest and most common choice is the same model that produced the simulation: the model that generated the behavior also certifies that the behavior is faithful. We measure what that choice misses, and we test whether capability rather than family explains it. On n=107 frozen generated responses scored under a support-in-context rubric, we run three judges under identical rubric, prompts, context, and decoding: a same-family self-judge (the 72B generator scoring itself), a matched-scale (~70B) judge from a distinct provider family, and a stronger frontier cross-family judge. The self-judge flags 6/107 unsupported outputs (5.6%); the matched-scale distinct-family judge, in the same capability class but a different family, flags 57/107 (53.3%). The disagreement is perfectly one-directional: 51 items the matched judge flags and the self-judge passes, 0 in reverse (McNemar p<1e-12). This asymmetry is consistent with a self-/family effect and rules out the reading that the self-judge is simply a weaker judge detecting less; one matched-scale judge cannot, on its own, separate that family component from a generic strictness difference between two models. Capability still adds on top: the frontier judge flags more again (92.5%). The cheapest default judge passes nearly every output a same-scale distinct-family judge reads as unsupported, exactly the model-bias artifacts a fidelity check exists to catch. We give a minimal audit protocol and argue that one-directional disagreement should function as a validation rule, not a tuning knob. We test one generator on a context-grounding proxy; transfer to live persona transcripts is the open boundary.