Emergent Misalignment as a Distributional Phase Transition in Rank-1 LoRA Fine-Tuning
Abstract
Fine-tuning large language models on narrowly harmful datasets can induce emergent misalignment (EM), where models exhibit broadly misaligned behaviors on unrelated prompts. Yet existing work primarily measures EM through aggregate misalignment rates and compares final fine-tuned models against base models, leaving the developmental dynamics and statistical structure of EM largely unexplored. In this work, we show that EM is better understood as a distributional phase transition during rank-1 LoRA fine-tuning of Qwen2.5-7B-Instruct. By scaling the B matrix at inference, amplifying its magnitude while preserving its direction, we reveal transition dynamics that are subtle at natural scale. Across domains, we identify a consistent transition point at which behavioral alignment scores shift from tight low-variance clusters to high-variance left-skewed spread, occurring simultaneously with a sharp spike in response activation variance. By framing EM as a distributional phase transition, we hope to move beyond post-hoc behavioral description towards a mechanistic, predictive understanding of misalignment development during fine-tuning.