Simple Activation Clipping Defeats Circuit Breakers

Recent works in LLM safety propose methods that can be baked directly into the model's weights, breaking LLM inference on harmful prompts. These methods allow for open-weight models to be released with baked-in safety guardrails that cannot be trivially switched off. In this work, we analyze the behavior of safety "circuit breakers" and their impact on model's internal representations. We use interpretability tools to view circuit breaker triggers in LLM activation space and find that they can be localized to a few interpretable directions. Using our findings, we propose a steering-adjacent technique that can override circuit-breaker directions by clipping model activations in localized directions.