DSpark Training Cost Calculator

Estimate what it costs to train a DSpark speculative-decoding drafter against a custom target. It shows the size of the target cache. It also shows the duration of the run, and whether the run fits in VRAM. It reads the target shape from a HuggingFace or ModelScope config. You can also paste a config.json or enter the values yourself.

Target source

Enter the target values yourself.

The width of the feed forward block.

This value sets the Markov head and the shared embedding.

Use 0 for a dense target.

The router top-k. The calculator ignores it on a dense target.

The width of one expert.

Training data

The recipe OpenBMB published for openbmb/MiniCPM5-2B-DSpark: 1,959,525 sequences of about 3,600 tokens, which is 7.05 billion tokens for each pass, read 6 times. That is 42.3 billion tokens of arithmetic over 5.7 times the unique data of the paper recipe. Their published drafter checkpoint is 323,776,001 parameters and reaches an acceptance length of 5.52 at temperature 0. One pass uses 7,054,290,000 tokens.

The sequences in the training set.

The paper trains each drafter for 10 epochs.

One pass holds 7,054,290,000 tokens, read 6 times. This product, and not the sample count alone, sets both the target cache and the duration.

Target supervision

The run sends the target through once and writes its hidden states to storage. Training then runs without the target. It needs little VRAM and a lot of storage.

Draft recipe

Each layer adds 1 hidden state for each token to the target cache. The published setting is 5. Fewer layers are the main way to make the target cache smaller.

The backbone depth. The published drafters use 5.

The tokens drafted in each block. This is gamma in the paper.

The blocks sampled from each sequence at each step. The run caps this at 1 block for each sequence token, so a short sequence needs fewer.

The rank of the sequential head. A rank of 0 disables it and leaves a fully parallel drafter.

The published recipe trains at a learning rate of 0.0006. It uses a 4 percent warmup and an effective batch of 512. Neither value changes the cost, so neither is an input here.

Hardware

The calculator reads the target weight format from its config. The drafter itself is always trained in bf16.

This GPU has 32 GB of VRAM and 419 TFLOPS dense in bf16. 32 GB GDDR7. 3352 AI TOPS.

The run divides across these GPUs, and so does the model state it holds. The reference configurations assume 1 node of 8.

The run reads the target cache back in each epoch. This setting therefore decides whether the GPU or the storage sets the bound.

The fraction of the peak.

The multiplier over the bound.

The sequences in flight on each card.

Estimated training run

21.4 days

Fits · RTX 5090

Training a DSpark drafter against this target takes about 21.4 days on RTX 5090. The target has 42 blocks and 2,048 hidden. The run is limited by the GPU that does the arithmetic.

Tokens for each epoch
7,054,290,000
Arithmetic alone
10.7 days
Target cache reads alone
5.3 hours
Training data on disk
26.28 GiB
Drafter parameters
323,750,144
Anchors for each sequence
514
Target parameters
2,516,582,400
Overhead factor
2x

How big is the target cache?

157.7 TiB of target hidden states

One token stores 24.01 KiB: 2,048 values from each of the 5 captured target layers in bf16. It also stores the last hidden state for the distribution and confidence losses. The token ids and 2 masks come after that. Over 7,054,290,000 tokens, that is 157.7 TiB. The run writes it once and reads it back in each epoch.

The training data needs 26.28 GiB of storage. That is the regenerated text the run reads its tokens from, and the target cache does not replace it.

Capture fewer target layers if this does not fit your storage. The target cache is close to proportional to the captured layer count.

How long does the run take?

10.7 days of arithmetic and 5.3 hours of target cache reads

Training scores 7,050,370,950 positions in each epoch against 323,750,144 drafter parameters. That is 8.22e+19 FLOPs. The context that each block attends to adds 6.24e+18. Building the target cache adds 2.80e+19. The 2x overhead and the setup apply on top of the slower value.

Does it fit in VRAM?

5.20 GiB peak against 32.00 GiB of VRAM on RTX 5090

The drafter and its optimizer state need 617.5 MiB and 2.41 GiB. The gradients need 617.5 MiB. The activation buffer needs 84.33 MiB. Offline capture keeps the target out of VRAM while the drafter trains, so the drafter and its state set the peak. That is why one GPU is enough.

This run fits. The target cache still needs 157.7 TiB of storage, and the target checkpoint needs its own space on top.

What does the drafter cost?

323,750,144 parameters, about 12.9 percent of the target

The backbone holds 235,929,600, and the projection from the captured target layers holds 20,971,520. The Markov head holds 66,846,720. The confidence head holds 2,304. The embedding and the language model head are shared with the target and frozen. The run never trains them.

At inference time, the drafter proposes 7 candidate tokens at each step. The frozen target verifies them in 1 parallel pass. The acceptance length depends on the training data and on the verification schedule you deploy. This calculator measures neither one.

Questions about DSpark

What does it cost to train a DSpark drafter?

It depends almost entirely on two choices: how many training tokens you use, and whether you precompute the target cache or capture it online. Offline training over the published recipe writes tens of terabytes of target hidden states but trains on one consumer card. Online capture writes nothing and needs the whole target model resident for the entire run. Enter a target above and the calculator answers both.

Why is the target cache so large?

The drafter is supervised on the target model itself, not just its output tokens, so training needs the target hidden states for every token in the training set. Each token stores one bf16 vector per captured layer, plus the last hidden state, plus its token id and two masks. Five captured layers on a 2560 wide model is about 30 kilobytes per token, which over a billion tokens is tens of terabytes.

How do I make the cache smaller?

Capture fewer target layers. The cache is proportional to the captured layer count, so going from five layers to two cuts it by roughly 60 percent. The paper found that a shallower drafter still beats a fully parallel one, so the layer count is a real tuning knob rather than a fixed requirement. A smaller training set is the other lever, at the cost of a weaker drafter.

Can I train DSpark on one GPU?

Yes, in offline mode, and that is the usual way. The drafter itself is small: five blocks of backbone plus a projection and two heads. The frozen target is the large part, and offline training runs the target once during cache preparation and then lets it go. Online capture needs the target resident for the whole run, which on a large model is the difference between one card and several.

How long does DSpark training take?

On the published recipe, with the cache on fast host memory, the arithmetic and the cache reads come out close to the same size, so the run takes a similar number of hours either way. Move the cache to a network share and the run becomes entirely I/O bound and takes days. The bound line above says which of the two your settings produce.

Do I need to regenerate the training data?

Yes, and it is not optional. Training is teacher forced, so the drafter learns to imitate the target distribution on the target own outputs. If you train against a dataset written by a different model, or by a different mode of the same model, the acceptance length at inference drops. Regenerate the answers with the exact target checkpoint you intend to serve.

Which target models can a DSpark drafter attach to?

Any decoder whose config exposes a hidden size, a depth, and a usable embedding and language model head, which in practice means any modern decoder. The published drafters cover Qwen3 and Gemma. A drafter is bound to the exact target weights it was trained against, so a fine tune, a merge, or a quantized variant needs its own drafter.

Does this estimate the serving speedup?

No. This calculator covers training only. The speedup you get at serving time is set by the acceptance length the drafter achieves, which depends on the data you trained on and the verification schedule you deploy. The paper reports 60 to 85 percent faster per user generation at matched throughput against its production baseline.