Polymathic AI and Princeton Researchers Release FlashMap, a Flow Map Drafter for 6.14x Faster LLM Decoding

Researchers from Polymathic AI, Princeton, USC ISI, NYU, and the Flatiron Institute released FlashMap, a speculative decoding drafter that proposes a 16-token block in a single pass. The target model checks the block and keeps what matches, so output is identical to what it would have produced alone. On Qwen3-8B, the team reports up to a 6.14x speedup on GSM8K.

Inference cost is where AI economics live now. Faster decoding with no quality loss means lower bills, snappier responses, and more room to run capable models on modest hardware.

We like this kind of release. It is open, it is concrete, and it attacks a real bottleneck. The weights are Apache-2.0, the code is MIT, and there are plugins for vLLM and SGLang, which means practitioners can actually try it rather than admire a chart.

The core idea is clever. Earlier one-pass drafters predict each slot in a block independently, which produces incoherent guesses that mix two plausible continuations into nonsense, and the target rejects them. FlashMap uses a continuous flow map so a single noise sample picks the whole block jointly, keeping the drafted tokens consistent with each other. Better coherence means more tokens accepted per check, which is where the speed comes from.

Now the honest fine print, which the coverage to its credit includes. The 6.14x is the best case: greedy decoding on a math benchmark. At temperature 1 on chat-style Alpaca prompts, the speedup drops to 2.28x. On the smaller Qwen3-4B, it trails the DFlash baseline on four of nine tasks. The comparisons use baselines the authors retrained themselves, on a single H200 at concurrency one. Production servers juggling many users at once may see different numbers. And the analysis shows that the one-step mode keeps only a small share of the joint-structure gain; the two-step mode recovers much more of it at some cost in speed.

So our read: a real, useful step, not a universal 6x. Speculative decoding gains stack up quietly across the industry, and each one makes useful AI cheaper to deliver. The downside is mostly indirect: cheaper inference also lowers the cost of spam and abuse at scale. That is not a reason to slow down research like this; it is a reason platforms cannot count on compute cost as a brake.

Speculative decoding pairs a small drafter model with a larger target model. The drafter proposes several tokens, the target verifies them in one parallel pass, and the longest matching prefix is kept, so output is unchanged. FlashMap's drafter is about 0.5 billion parameters, reads hidden states from the target model, and was trained on 500,000 samples, with roughly 4,600 H200 GPU hours spent across all models and ablations.