ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference

Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with sparse attention and verifying them with full attention, but existing batched methods remain \emph{synchronized}: all requests in a batch share a single draft-verify schedule, even though the optimal draft length varies widely across requests and changes dynamically within each request. We propose \textbf{ASPIRE}, a non-synchronized batched self-speculative decoding framework built on three components. First, a \emph{unified mixed forward} allows drafting and verifying requests to coexist in the same batched forward pass, removing the need for global draft-verify phases. Second, a lightweight \emph{online speculation scheduler} uses per-request acceptance-rate estimates and a batch-aware cost model to let each request independently choose when to verify. Third, an \emph{intra-draft refresh layer} performs full attention at a single designated layer during drafting, updating the sparse context at every draft step so that it does not become stale. Across three models and five reasoning and long-context benchmarks, ASPIRE achieves $1.70$-$4.58\times$ decoding throughput over autoregressive baselines and consistently outperforms prior self-speculative methods. To promote reproducibility, we will release our code upon acceptance.