HarmThoughts: A Benchmark for Fine-Grained Harmful Behavior Detection in Reasoning Traces

Large reasoning models (LRMs) produce complex and multi-step reasoning chains, yet safety evaluation remains focused on final outputs. When reasoning models are jailbroken, harm does not appear all at once -- instead it propagates through distinct behavioral steps such as suppressing refusal, rationalizing compliance, decomposing harmful tasks, and concealing risk. No existing benchmark captures this process at the granularity needed to develop and evaluate step-level safety monitoring. To address this, we introduce HarmThoughts, a benchmark for step-wise safety evaluation of reasoning traces. HarmThoughts is designed around our proposed harm taxonomy encompassing 16 harmful reasoning behaviors across four functional groups that characterize how harm propagates rather than what harm is produced. The dataset consists of 56,931 sentences across 1,018 reasoning traces from four model families, each annotated with fine-grained sentence-level behavioral labels. Using HarmThoughts, we analyze harm propagation patterns across reasoning traces, identifying common behavioral trajectories and drift points where reasoning transitions from safe to unsafe. Finally, we perform a systematic study on the capability of white-box and black-box detectors on the task of identifying behaviors encompassed by HarmThoughts on this benchmark. Our results show that existing methods struggle with fine-grained behavior detection in reasoning chains, particularly for nuanced categories within harm emergence and execution, emphasizing the need for stronger process-level monitoring tools as reasoning models become more capable.