AI News
All stories
Reported

Tencent releases DSpine draft model for Qwen3.8-27B speculative decoding

Tencent released DSpine, a five-layer speculative draft model for Qwen3.8-27B, and reports 2.9 to 3.8 times throughput over autoregressive decoding.

Open Source · · 2 min read

Tencent published tencent/Qwen3.8-27B-DSpine-b16 on Hugging Face, a five-layer draft model meant to speed up Qwen3.8-27B. The model uses a block size of 16 and an 8K sliding window. Tencent reports roughly 2.9 to 3.8 times throughput compared with autoregressive decoding, which produces one token at a time, in SGLang 0.5.22 evaluations. It also says DSpine beats DFlash2 on accepted draft length and agent throughput.

Speculative decoding works by letting a small draft model propose several tokens that the large model then checks. The large model can accept a run of tokens in one pass instead of generating each one separately. According to a r/LocalLlama recap of the release, DSpine drafts tokens in parallel rather than in sequence. It injects predicted features from earlier predicted tokens across depth so that the dependencies between tokens are preserved. A linked paper thread titled "Draft in Parallel, Condition Through Depth" names the mechanism Adjacent Causal Injection. Tencent's evaluations used BF16 precision, FlashAttention 3, dynamic verification and ReplaySSM. The recap does not describe the prompts or hardware behind the 2.9 to 3.8 times range, so that figure rests on Tencent's own evaluation setup.

Early third-party tests point in different directions. A commenter who ran the MegaCapybara engine on an RTX 5090 with roughly Q5 weights measured no-thinking coding speed rising from 420 tok/s with DFlash2 to 800 tok/s with DSpine. Reasoning output fell in the same test, from 320 tok/s to 280 tok/s. The commenter attributed that drop to the first one or two draft tokens being predicted with lower confidence, which they suggested could be recoverable with more training. A separate tester who added LumaBrowser support for llama.cpp reported that DSpine was not faster than DFlash2 in any scenario they tried and used more memory. They said the outcome might differ in SGLang, but described their llama.cpp results as showing "zero upsides."

Commenters in the same thread tied the release to an expected Qwen4 and speculated about further gains. One estimated a 15 to 20% improvement over DFlash2 and expected more after additional training and engine-side optimization. Those are the commenter's own projections, not measured results. The mixed early reports come from different engines, and the recap does not show that any of them used the same workload as Tencent's SGLang evaluations.

What changed

Tencent released tencent/Qwen3.8-27B-DSpine-b16, a five-layer speculative draft model for Qwen3.8-27B.

Type: New capability · Applies to: tencent/Qwen3.8-27B-DSpine-b16, Qwen3.8-27B

Why it matters

Speculative decoding gains depend heavily on workload and inference engine, so teams should test drafts on their own tasks before adopting them.

Original source

We have no link to the original source for this story yet.

Where we saw it

1 source

Latent Space

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works