Paper 2605.29707
Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding
- Published
- May 2026
- Research lab
- Independent
- Citations
- 8
- GitHub
- 128 stars
01 In brief
Summary
Domino is a speculative decoding framework that decouples causal dependency modeling from expensive autoregressive draft execution.
It uses a parallel draft backbone (based on DFlash) to generate preliminary draft distributions for the entire block, then applies a lightweight Domino head—comprising a GRU causal encoder and a low-rank correction head—to refine these distributions with prefix-dependent causal information.
Training uses teacher forcing for the causal encoder and a base-anchored curriculum that first strengthens the parallel backbone and then gradually shifts optimization toward the final causally corrected distribution.
Experiments on Qwen3-4B and Qwen3-8B show Domino achieves up to 5.49× end-to-end speedup under the Transformers backend and up to 5.8× throughput speedup under SGLang serving, outperforming EAGLE-3, DART, and DFlash.
Ablations confirm that teacher forcing and the curriculum improve acceptance length, and the Domino head adds only 56M parameters (+5.3%) with a 2.8% latency increase while improving acceptance length by 16.6% and speedup by 12.3% over DFlash.
The method is limited to SGLang and may require platform-specific optimization for other serving frameworks or hardware.
02 From the paper
Abstract
Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel with the target model. However, its practical speedup is constrained by the trade-off between draft quality and drafting cost: autoregressive drafters model causal dependencies among draft tokens but incur sequential overhead, while parallel drafters reduce drafting cost but weaken intra-block dependency modeling. In this paper, we propose Domino, a speculative decoding framework that decouples causal dependency modeling from expensive autoregressive draft execution. Domino first uses a parallel draft backbone to produce preliminary draft distributions for the entire block, and then applies a lightweight Domino head to refine them with prefix-dependent causal information. To stabilize teacher-forced causal encoding, we further introduce a base-anchored training curriculum that first strengthens the parallel backbone and then gradually shifts optimization toward the causally corrected final distribution. Experiments on Qwen3 models show that Domino achieves up to \(5.49\times\) end-to-end speedup under the Transformers backend and up to \(5.8\times\) throughput speedup under SGLang serving.