The Year in AI Papers
The year/DeepSeek

Paper 2607.05147

DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

Xin Cheng, Xingkai Yu, Chenze Shao, Jiashi Li, Yunfan Xiong, Yi Qian, Jiaqi Zhu, Shirong Ma, Xiaokang Zhang, Jiasheng Ye, Qinyu Chen, Chengqi Deng, et al.

Published
Jul 2026
Research lab
DeepSeek
Citations
12
GitHub
6.9K stars

01 In brief

Summary

DSpark is a speculative decoding framework that combines a semi-autoregressive draft model with confidence-scheduled verification to accelerate LLM inference.

The draft model uses a parallel backbone (based on DFlash) to generate tokens in one pass, followed by a lightweight sequential head (Markov or RNN) to model inter-token dependencies, mitigating suffix decay.

A confidence head estimates per-position acceptance probabilities, calibrated via Sequential Temperature Scaling, and a hardware-aware prefix scheduler dynamically selects verification lengths per request to maximize system throughput based on real-time load.

Offline evaluations on Qwen3-4B/8B/14B and Gemma4-12B show DSpark improves accepted length by 16-30% over DFlash and 26-31% over Eagle3.

In production deployment with DeepSeek-V4, DSpark accelerates per-user generation by 60-85% (Flash) and 57-78% (Pro) at matched throughput, and maintains robust throughput under strict SLA constraints where the MTP-1 baseline degrades.

The framework is open-sourced with checkpoints and the DeepSpec training repository.

02 From the paper

Abstract

Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.