LOCUS: Head Selection and Subspace Projection for Targeted Activation Steering.

Activation steering is a lightweight paradigm for controlling large language models, modifying hidden representations at inference time without updating weights. Standard difference-of-means approaches estimate a per-layer steering direction from contrastive data and broadcast it across the full residual stream, which may couple the intervention to off-target axes present in the contrastive data and degrade unrelated capabilities. To mitigate this issue, we introduce LocUS (Localized Unembedding Steering), which uses the model's own unembedding matrix to refine the scope of the intervention. From a property-relevant set of unembedding columns, LocUS selects the attention heads whose outputs are most compactly reconstructed in that subspace, and restricts the steering update at each selected head to a specific, sparse set of property-aligned directions chosen automatically. Across three model families, LocUS matches or improves on standard difference-of-means for toxicity mitigation and sentiment redirection while intervening on under 6% of attention heads and better preserving general capability control metrics.