Computer Use with Evolving Software

Computer Use Agents (CUAs) act through an interface that is designed by hand and fixed before any task begins, comprising a fixed observation structure, a curated action space, and a workflow that routes between them. This interface commits every task to one workflow and bounds what the agent can learn, since any procedure it acquires is limited to the action vocabulary the interface already provides. We argue that this interface need not be fixed at all. The model can instead write it for each task in code, defining its own observation structure, control flow, and action space as programs it runs at inference time. We find that a model writing its own interface this way, with no computer-use-specific engineering, is already competitive with specialised computer-use scaffolds on OSWorld. Building on this, we introduce CUES-Evol, an evolutionary procedure that constructs a persistent repository of reusable software the model draws on as it discovers each task's interface. A meta coding agent grows this repository from agent rollouts alone, with no human in the loop, repairing failures and accumulating new abstractions across tasks. On ScienceBoard, a benchmark of scientific computer-use tasks, the evolved repository surpasses the hand-designed Agent S3 across three models, raising pass rate from 22.0\% to 30.9\% on Haiku, 30.1\% to 40.4\% on Sonnet, and 18.4\% to 28.0\% on Qwen3.5, while cutting median token cost by roughly half.