Particles of Thought: Aligning Process Supervision with Test-Time Search

Large language models have demonstrated strong mathematical reasoning, yet effective test-time scaling remains challenging. Automated process supervision methods train value functions on unguided rollouts, but inference-time search selectively retains high-value states, creating a distribution mismatch that degrades guidance quality. We propose Particles of Thought (PoT), a framework that aligns process supervision with test-time search. PoT formulates reasoning as probabilistic inference, using Sequential Monte Carlo at both training and test time to approximate the posterior over correct reasoning paths. We train our guidance model on particles sampled from an Oracle SMC process that filters based on rollout success; at test time, this learned guidance model guides the search, aligning training and inference distributions. Empirically, Guided SMC is the strongest test-time decoder at a matched sampling budget, outperforming Best-of-N and beam search on Math500 and OlympiadBench. Next, any gain from distribution-matched guidance falls within the standard deviation across seeds, so distribution matching gains are inconclusive. We further find that roughly half of outcome-correct solutions do not faithfully derive their answer, which may hinder learned guidance and limit its transfer to a different policy family.