EnterpriseGuard: A Counterfactual Test of Guardrail Adherence in Collaborative Enterprise Agents
Abstract
Enterprise AI agents in tools like Slack and Copilot are evaluated mostly on task success, yet an agent that completes a task by bypassing an access control looks identical, on a success metric, to one that stays within policy. This equifinality is exactly what internal audit exists to surface: auditors test how a control behaves under adversarial conditions, not just whether an outcome occurred. We propose EnterpriseGuard, a counterfactual behavioral benchmark that instruments a simulated enterprise chat environment and varies requester authority, social-engineering pressure, and conflicting multi-user instructions while holding the legitimate task fixed. Rather than a pass/fail compliance score, it recovers what the agent did, when adherence broke, why (the strategy in its action trace), and how adherence changes across conditions, treating the invariance of a guardrail, not its average level, as the behavioral property. The failure taxonomy is grounded in real internal-audit control domains, giving deployers a repeatable behavioral assurance instrument.