What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce
Abstract
Online marketplaces are beginning to be mediated by autonomous AI agents that act on behalf of consumers: rather than humans browsing and clicking, agents parse webpages or query APIs to view, evaluate, and choose products. We study how these agents shop using ACES, a provider-agnostic sandbox that pairs a browser-use agent with a fully programmable mock marketplace and recovers causal choice responses through randomized experiments. Auditing three frontier agents (Claude Opus 4.5, GPT-5.1, and Gemini 3 Pro Preview), we find that agentic demand looks fundamentally different from human-centric commerce. Agents exhibit pronounced choice homogeneity, concentrating demand on a few modal products while leaving some brands essentially never selected. These preferences are strikingly model-dependent: the same query yields entirely different winners across agents. Agents respond sharply to where a product sits on the page, but which positions are valuable varies by model---one agent places nearly all weight on the top row, another inverts the convention and favors the bottom---so there is no universal notion of a top rank, and these position biases persist even in text-only headless API interfaces and under instructions to ignore them. Agents consistently penalize sponsored tags while rewarding platform endorsements, and sensitivities to price, ratings, and reviews differ widely across models. Finally, sellers can respond: a seller-side agent that makes simple, query-conditional description edits can drive large market-share gains. These findings highlight that agentic markets are volatile and exploitable, motivating continuous auditing and raising new questions for platform design, seller strategy, and regulation.