Linear DSA: Budget-Aware Dynamic Sparse-to-Linear Atten- tion via Priority-Scored KV Cache Eviction

Token-level dynamic sparse attention (DSA), exemplified by DeepSeek's Lightning Indexer, achieves superior model quality via an exact Top-$K$ selection over key-value tokens. However, both the indexer computation and the KV cache storage grow linearly with sequence length, creating a scalability bottleneck for long-context inference. Existing KV cache compression methods require careful, model-specific tuning and add significant algorithmic complexity. We observe that DSA itself naturally provides a high-quality access signal for eviction: tokens recently selected by the Top-$K$ indexer are precisely those the model deems important. Leveraging this insight, we propose \emph{Linear DSA}, a simple eviction policy that maintains a fixed KV cache budget by evicting the least-recently-accessed token as determined by DSA's own selection pattern. Without any retraining or architectural modification, Linear DSA on GLM-5.1 matches full-attention performance on AIME-25/26 at a budget of 80K, while substantially outperforming both StreamingLLM and random eviction across moderate budgets. These results suggest that DSA's implicit utility extends beyond attention sparsification, and it simultaneously serves as a built-in importance signal that enables trivial, yet highly effective KV cache management.