Recent advancements have shown that reinforcement learning (RL) can substantially improve the reasoning abilities of large language models (LLMs). The effectiveness of such RL training, however, depends critically on the exploration space defined by the pre-trained model's token-output distribution. In this paper, we revisit the standard cross-entropy loss, interpreting it as a specific instance of policy gradient optimization applied within a single-step episode. To systematically study how the pre-trained distribution shapes the exploration potential for subsequent RL, we propose a generalized pre-training objective that adapts on-policy RL principles to supervised learning. By framing next-token prediction as a stochastic decision process, we introduce a reward-shaping strategy that explicitly balances diversity and precision. Our method employs a positive reward scaling factor to control probability concentration on ground-truth tokens and a rank-aware mechanism that treats high-ranking and low-ranking negative tokens asymmetrically. This allows us to reshape the pre-trained token-output distribution and investigate how to provide a more favorable exploration space for RL, ultimately enhancing end-to-end reasoning performance. Contrary to the intuition that higher distribution entropy facilitates effective exploration, we find that imposing a precision-oriented prior yields a superior exploration space for RL.

Reasoning models think in long, unstructured streams with no mechanism for compressing or organizing their own intermediate state. We introduce MEMENTO: a method that teaches models to segment reasoning into blocks, compress each block into a \emph{memento}, i.e., a dense state summary, and reason forward by attending only to mementos, reducing context, KV cache, and compute. To train Memento models, we release OpenMementos, a public dataset of 228K reasoning traces derived from OpenThoughts-v3, segmented and annotated with intermediate summaries. We show that a two-stage SFT recipe on OpenMementos is effective across different model families (Qwen3, Phi-4, OLMo3) and scales (8B--32B parameters). Trained models maintain strong accuracy on math, science, and coding benchmarks while achieving ${\sim}2.5\times$ peak KV cache reduction. We extend vLLM to support our inference method, achieving up to $2\times$ throughput improvement while also enabling us to perform RL and further improve accuracy. Finally, we identify a \emph{dual information stream}: information from each reasoning block is carried both by the memento text and by the corresponding KV states, which retain implicit information from the original block. Removing this channel drops accuracy by 15pp on AIME24.

Evaluation benchmarks play a central role in the development and governance of AI systems, yet persistent concerns about val concerns about their validity remain. It is unclear that such benchmarks actually measure the concepts they claim to assess. Drawing inspiration from the principles of $\textit{convergent and discriminant validity}$, we analyze 56 safety and capability benchmarks across 53 models using correlation analyses and item-level prediction models. When studying benchmarks purporting to measure the same construct, we often see inconsistent consistent rankings on the models. This suggests that the constructs, such as safety detection and exaggerated refusal, may not be sufficiently well-defined. When comparing across constructs, we find that some concepts are distinguishable from others (e.g., unsafe generation is different from bias), but many concepts (e.g., reasoning, knowledge, and comprehension) are not well separated. In some cases, evaluation format and task structure drive correlations between benchmarks more than the concepts they aim to measure. Finally, we investigate individual benchmarks to identify benchmarks that likely measure a different concept than reported. In particular, we find that BBQ, one of the most widely used bias benchmarks, is more correlated with reasoning benchmarks than with other bias benchmarks. Overall, we encourage benchmark developers and practitioners to evaluate benchmarks through the lens of convergent and discriminant validity when developing and using benchmarks. We release our extensive item-level dataset of model predictions to support future empirical work on benchmark validity.

Diffusion language models (DLMs) offer a compelling promise: parallel token generation could break the sequential bottleneck of autoregressive (AR) decoding. Yet in practice, DLMs consistently lag behind AR models in quality. We argue that this gap stems from a fundamental failure of introspective consistency: AR models agree with what they generate, whereas DLMs often do not. We formalize this via the introspective acceptance rate, which quantifies whether a model internally accepts its previously generated tokens. Through this lens, we uncover a key structural advantage of AR models: causal masking combined with logit shifting implicitly enforces introspective consistency during training. Motivated by this insight, we introduce the Introspective Diffusion Language Model (I-DLM), a new paradigm that preserves the introspective consistency of AR training while retaining the diffusion-style parallelism. I-DLM uses a novel introspective strided decoding (ISD) algorithm, which enables the model to verify previously generated tokens while advancing new ones in the same forward pass. This yields a new quality–efficiency frontier unavailable to either AR or prior diffusion models, with stride providing a controllable tradeoff between verification depth and parallel progress. Empirically, I-DLM-8B is the first DLM to match the quality of its same-scale AR counterpart while surpassing all prior DLMs in both quality and practical serving efficiency across 15 benchmarks. It attains 72.5 on AIME-24 and 45.1 on LiveCodeBench-v6, outperforming LLaDA-2.1-mini (16B) by more than 29 and 14 points, respectively. Finally, with a single-pass self-speculative decoding pipeline and gated LoRA, ISD enables lossless acceleration and better efficiency at high concurrency compared to speculative decoding.