ai.hanz.devSource

DSpark Training Cost Calculator

Estimate what it costs to train a DSpark speculative-decoding drafter against a custom target model: how large the target hidden state cache gets, how long the run lasts, and whether it fits in VRAM. The target shape is read from a HuggingFace or ModelScope config, a config.json you paste, or numbers you type.

Target source

Describe the target by hand.

Dense feed forward width.

Sets the Markov head and the shared embedding.

Zero for a dense target.

The router top-k. Ignored on a dense target.

Width of one expert.

Training data

The recipe OpenBMB published for openbmb/MiniCPM5-2B-DSpark: 1,959,525 sequences read six times, which is 7.05 billion tokens, or five times the unique data of the paper recipe. Their published draft checkpoint is 323,776,001 parameters and reaches an acceptance length of 5.52 at temperature 0. 1,175,715,000 tokens in one pass.

Sequences in the training set.

The paper trains each drafter for 10.

1,175,715,000 tokens in one pass, read 6 times. This product, not the sample count alone, is what drives both the cache and the run.

Target supervision

Runs the target once to write its hidden states to disk, then trains without it. Cheap in VRAM, very expensive in storage.

Draft recipe

Each one adds a hidden state per token to the cache. Five is the published setting, and fewer is the main way to shrink the cache.

Backbone depth. Five matches the published drafters.

Tokens drafted per block, the gamma of the paper.

Blocks sampled from each sequence per step. Capped at one block per sequence token, so a short sequence needs fewer.

Rank of the sequential head. Zero disables it, leaving a purely parallel drafter.

The published recipe trains at a learning rate of 0.0006 with a 4 percent warmup and an effective batch of 512. Neither affects the cost, so neither is an input here.

Hardware

32 GB of memory, 419 TFLOPS dense in bf16. 32 GB GDDR7. 3352 AI TOPS.

The run shards across these. The reference configurations assume a single node of eight.

The cache is read back once per epoch, so this decides whether the run is bound by the GPU or by the disk.

Fraction of peak.

Multiplier over the bound.

Sequences in flight.

Estimated training run

3.4 days

Fits · RTX 5090

Training a DSpark drafter against this target with 42 blocks and 2,048 hidden takes about 3.4 days on RTX 5090, and it is limited by the GPU doing the arithmetic.

Tokens per epoch
1,175,715,000
Compute alone
40.7 hours
Cache read alone
53 minutes
Draft parameters
323,750,144
Anchors per sequence
85
Target parameters
2,516,582,400
Overhead applied
2x

How big is the target cache?

26.29 TiB of target hidden states

Each token stores 24.01 KiB: 2,048 values from each of the 5 captured target layers in bf16, the last hidden state that the distribution and confidence losses both need, the token ids, and two masks. Over 1,175,715,000 tokens that is 26.29 TiB written once, and read back once per epoch.

Capturing fewer target layers is the first lever if this does not fit your disk. The cache is close to proportional to the captured layer count.

How long does the run take?

40.7 hours of arithmetic, 53 minutes of reading the cache back

Training scores 1,165,917,375 positions per epoch against 323,750,144 draft parameters, which is 1.36e+19 FLOPs, plus 1.72e+17 for the context each block attends to and 4.66e+18 to build the cache. The 2x overhead and setup land on top of the slower of the two.

Does it fit in VRAM?

5.13 GiB peak against 32.00 GiB on RTX 5090

The drafter and its optimizer state are 617.5 MiB and 2.41 GiB, gradients are 617.5 MiB, and the activation buffer is 13.95 MiB. Offline capture means the target is not resident while the drafter trains, which is what makes one card enough.

This run fits. The cache still needs 26.29 TiB of storage, and the target checkpoint needs its own space on top.

What does the draft model cost?

323,750,144 parameters, about 12.9 percent of the target

235,929,600 in the backbone, 20,971,520 in the projection from the captured target layers, 66,846,720 in the Markov head, and 2,304 in the confidence head. The embedding and the language model head are shared with the target and frozen, so they are never trained.

At inference the drafter proposes 7 candidate tokens per step, which the frozen target verifies in a single parallel pass. The acceptance length you actually get depends on the data you trained on and on the verification schedule you deploy, neither of which this calculator measures.

Questions about DSpark

What does it cost to train a DSpark drafter?

It depends almost entirely on two choices: how many training tokens you use, and whether you precompute the target cache or capture it online. Offline training over the published recipe writes tens of terabytes of target hidden states but trains on one consumer card. Online capture writes nothing and needs the whole target model resident for the entire run. Enter a target above and the calculator answers both.

Why is the target cache so large?

The drafter is supervised on the target model itself, not just its output tokens, so training needs the target hidden states for every token in the training set. Each token stores one bf16 vector per captured layer, plus the last hidden state, plus its token id and two masks. Five captured layers on a 2560 wide model is about 30 kilobytes per token, which over a billion tokens is tens of terabytes.

How do I make the cache smaller?

Capture fewer target layers. The cache is proportional to the captured layer count, so going from five layers to two cuts it by roughly 60 percent. The paper found that a shallower drafter still beats a fully parallel one, so the layer count is a real tuning knob rather than a fixed requirement. A smaller training set is the other lever, at the cost of a weaker drafter.

Can I train DSpark on one GPU?

Yes, in offline mode, and that is the usual way. The drafter itself is small: five blocks of backbone plus a projection and two heads. The frozen target is the large part, and offline training runs the target once during cache preparation and then lets it go. Online capture needs the target resident for the whole run, which on a large model is the difference between one card and several.

How long does DSpark training take?

On the published recipe, with the cache on fast host memory, the arithmetic and the cache reads come out close to the same size, so the run takes a similar number of hours either way. Move the cache to a network share and the run becomes entirely I/O bound and takes days. The bound line above says which of the two your settings produce.

Do I need to regenerate the training data?

Yes, and it is not optional. Training is teacher forced, so the drafter learns to imitate the target distribution on the target own outputs. If you train against a dataset written by a different model, or by a different mode of the same model, the acceptance length at inference drops. Regenerate the answers with the exact target checkpoint you intend to serve.

Which target models can a DSpark drafter attach to?

Any decoder whose config exposes a hidden size, a depth, and a usable embedding and language model head, which in practice means any modern decoder. The published drafters cover Qwen3 and Gemma. A drafter is bound to the exact target weights it was trained against, so a fine tune, a merge, or a quantized variant needs its own drafter.

Does this estimate the serving speedup?

No. This calculator covers training only. The speedup you get at serving time is set by the acceptance length the drafter achieves, which depends on the data you trained on and the verification schedule you deploy. The paper reports 60 to 85 percent faster per user generation at matched throughput against its production baseline.