When the Loop Closes Quietly: Mechanistic Interpretability as Contestability Infrastructure for Deployed AI
Abstract
By every local metric, an AI decision-support system deployed across six counties in a sub-Saharan African country was a success: faster, more accurate, and more consistent. It was also a closed loop. Operators had stopped contesting outputs they could not evaluate, and affected communities had stopped contesting decisions they could not trace to any human decision-maker. This paper uses that field case to examine what it would take for mechanistic interpretability to be actionable in deployed systems. I argue that interpretability becomes actionable when it functions as contestability infrastructure: the technical substrate that makes particular outputs challengeable by the officials, experts, and communities they affect. The paper maps observed contestability failures to interpretability primitives, including circuit analysis, feature attribution, sparse autoencoders, and activation monitoring, and argues that these methods provide technical preconditions for tracing, evaluating, and challenging AI-assisted decisions. The paper also marks the limits: interpretability cannot provide standing, access, institutional capacity, or resolution of underlying value disputes. I conclude with four testable predictions about the relationship between interpretability, operator deference, and contestability in deployed systems. The central claim is that mechanistic interpretability is necessary but not sufficient for contestability, and that treating interpretability as deployment infrastructure rather than solely as scientific inquiry offers a more concrete account of what makes it actionable.