Optimizing Behavioral Experiment Design with In-Silico Simulations

Running behavioral experiments is expensive, slow, and vulnerable to design flaws detectable only after data collection. We introduce a preliminary multi-agent framework that uses simulations to iteratively refine behavioral study designs before recruiting human participants. With personas instantiated through target demographic attributes, we elicit explanatory reasoning traces from LLM simulations that serve as diagnostic evidence of structural flaws. In two survey case studies, simulation-rich Critic outperforms a design-only baseline on the precision of high-severity structural errors, while providing little value for identifying missed opportunities. These early results suggest reasoning traces can expose a qualitatively distinct class of design flaws inaccessible to the design-only baseline.