Position: Approval Can Reinforce Misalignment in Human Oversight of Computer-Use Agents
Abstract
Approval-centered oversight has become one of the central safety mechanisms for computer-use agents (CUAs). We argue that when approval is uninformed and later operationalized as evidence of acceptability, this mechanism can not only fail to prevent misalignment but actively reinforce it. We call this dynamic cyclical misalignment: users infer from fluent agent behavior that the agent shares their attention, assumptions, and norms, while agent systems may treat approval as evidence that the action reflected informed user intent. These reciprocal inferences compound rather than self-correct. Repeated approvals can deepen users' mistaken mental models and, when carried forward through always-allow memory, preference learning, or escalation policies, reduce the likelihood of future intervention. For privacy and security, this cycle creates silent norm violations and adversarial exploitation surfaces under the formal appearance of oversight. Drawing on empirical studies of GUI agents in adversarial web environments, we ground this cycle in concrete cases and propose four design primitives for scaffolding epistemic agency: context negotiation, evidence-bearing approval, divergence-triggered escalation, and structured repair feedback.