Back to the Future: A workbook time machine for spread- sheet creation benchmarks

We introduce the workbook time machine, a pipeline that automatically creates benchmarks evaluating the ability of language models to create derived objects in spreadsheets (formulas, charts, pivot tables, and conditional formating). Applied to public workbook corpora, it produces WTM-CORPUS–a collection of (input workbook, output workbook, query) triples spanning four artifact types and varying complexity. From this corpus we curate WTM-BENCH, a 150-task evaluation benchmark with queries at three levels of specificity. We evaluate existing spreadsheet manipulation agents and baselines on WTM-BENCH across artifact types, step complexity, and instruction granularity. Our evaluations show that query specificity, agent orchestration and interface API used to control spreadsheets play a big role in LLM performance on Excel tasks.