The Compute Floor of Data Mixing: When Loss-Based Reweighters Start to Drain Easy Synthetic Text

Automatic data-mixing methods are usually studied with pretraining sweeps from 70M to 7B parameters, which puts even the question of how they differ behind a large compute budget. We ask whether a difference that separates the methods is already visible at a scale cheap enough to run on a single free-tier cloud GPU, at one model size and one pool. The diagnostic is simple: run candidate methods on a pool of 9 real Pile subsets and 9 synthetic instruction corpora, give each the same per-domain validation signal, and watch how it moves the synthetic share. At 160M parameters and about $2.30\times10^{17}$ FLOPs per run we see a clean directional split that holds across three seeds: loss-monotone reweighters (Group-DRO and a DoReMi-style variant) push the synthetic share down from 0.50 to 0.20 and 0.37, while three interaction or gradient estimators stay near the 0.50 start, Aioli and Skill-It tightly and DoGE only on the mean (its per-seed share swings from 0.453 to 0.557). The cause is a training-induced loss gap: the two provenances start nearly tied on the base checkpoint (within 0.04 nats), but within the first adaptation round the real Pile text becomes the hard (high-loss) provenance and the templated synthetic text the easy one, so any update monotone in a domain's own loss gets pulled toward real. We did not run a static retrain, so the only quality read is a held-out test loss on the online-trained models themselves; at $n=3$ it gives the method that moves least (Aioli) the lowest mean, beating only DoGE and Skill-It and within seed scatter of Stratified, DoReMi, and Group-DRO, an average-case signal rather than a ranking. We release per-run FLOP accounting, three tiered configs, and every CSV and figure, and frame the protocol as a behavioral diagnostic of which mixture each policy chooses, not a ranking of which trains a better model.