Optimal Context Stopping for Large Language Models

Large language models face a fundamental tension at inference time: additional context supplies task-relevant evidence, yet longer inputs induce $\textit{context rot}$---the degradation of model performance with growing input length. This trade-off arises in long-context modeling, repository-level coding, and language model agents. We ask: how much context should a model read before generating? We formalize this as the $\textit{Optimal Context Stopping}$ problem through the lens of statistical decision theory. Across diverse benchmarks, evaluation formats, and LLM families, we observe a non-monotonic relationship between performance and context length: accuracy initially improves but eventually declines beyond an optimum that varies across models and tasks, a phenomenon we term $\textit{context saturation}$. We decompose this behavior into two latent processes: a task-side information gain state and a model-side context rot state, and validate the decomposition with controlled experiments that measure each state in isolation. Context rot follows a regular quantitative law, and an interior optimum exists whose location is predicted by the measured gain and rot components. Our results advance the analysis of LLMs' macroscopic behavior and provide a theoretical framework for studying long-context understanding under bounded model capacity.