Inference-Time Scaffolding for Small-Model Math Reasoning: It's Not the Tools

Wrapping a 2B math LM in a two-tool agent harness (Python + inversion validator) lifts GSM8K accuracy by +7.58pp, and the field reads this as a tool-augmentation win. We ablate each component and find no measurable accuracy benefit from the tools in our main settings, and that on off-the-shelf checkpoints the harness is a net negative against plain chain-of-thought. Concretely: (1) On the paper's trained 2B+OPD+GRPO checkpoint (GSM8K n=1319), removing the Python tool, the validator, or both changes accuracy by <=0.53pp (<1 sigma). The tools are redundant Python channels. (2) Across seven model x dataset settings (Qwen2.5-7B-Instruct on GSM8K/SVAMP/MATH-500, Llama-3.2-3B-Instruct on GSM8K, Qwen3.5-2B+OPD+GRPO on GSM8K/SVAMP/MATH-500) the no-tools arm matches or beats the full harness (Figure 1). Camera-ready 5-seed reruns give +6.41 +/- 1.19pp on Qwen7B (5/5 seeds McNemar p < 10^-3) and +21.84 +/- 2.99pp on Llama-3B (5/5 seeds p < 10^-6). On the Llama-3.2-3B-Instruct row, plain single-shot CoT is 78.01%: the paper's full harness at 41.70% (Table 4) is a -36pp regression against plain CoT (our 5-seed repro at 31.76% widens the gap to -46pp), and -no-tools recovers only about half. (3) Efficiency: the no-tools arm uses 68.6% of the tokens and 30.7% of the turns of the full harness while beating it on accuracy. (4) An 80-token post-PASS nudge does help, but only checkpoint-specifically: -4.25pp on our 2B+OPD+GRPO checkpoint that enters a post-PASS "double-check spiral"; effect not distinguishable from zero on either off-the-shelf substitute over 5 seeds. A binary-Oracle-EFC proxy on 1950 pooled traces gives a monotonic Q1->Q4 accuracy rise of +6.98pp with raw tokens negatively correlated with success (r_pb=-0.21), consistent with the EFC view of Zhang et al. (2026) that additional raw compute need not translate into useful retained feedback. The field's vocabulary of "tool-augmented reasoning" attributes the lift to the wrong component.