Zero-Shot Classification Reachability Leaderboard

It is the smallest card in the hardware directory on this site that can hold the whole run, the number of cards here that can hold it, and the decode rate that configuration reaches. The run is the weights, the KV cache at the chosen context length and sequence count, an activation buffer, and a framework reserve.

The model list, the Decision Index, and the measured latency come from the clef-evals Decision Model Leaderboard. The leaderboard is scored on one RTX PRO 6000, so its latency figure is not the decode rate shown here.

The latency on the leaderboard is one end to end call, which includes prefill, the decision head, and the client round trip. The rate here is a bandwidth roofline for decode alone, which is the highest rate the card can reach once the prompt is read.

Either the checkpoint publishes no config this site can read, or the config describes an architecture that is not a decoder transformer. A closed hosted API publishes no checkpoint. An encoder only classifier publishes one, but it has no KV cache and no decode loop, so the sizing model does not apply to it.

It is config arithmetic rather than a measurement. The calculator counts the attention blocks, the dense feed forward, the expert bank, the router, and the embedding tables from the published config. It does not count a multimodal vision tower, a multi token prediction block, or the savings of layers that share weights, so a model with any of those can be a few percent off in either direction. The breakdown shows the parameter count the config produced beside the count the leaderboard reports.

The bytes for each weight follow the format. A checkpoint published in BF16 costs two bytes for each weight, while an FP8 or a four bit copy costs one byte or half a byte. Forcing a narrower format stores the same parameter count in fewer bytes, so smaller cards fit.

The KV cache grows with the context length and the sequence count, and it has to be resident beside the weights. A model that fits on a small card at a short context can need a larger card at a long one. The activation buffer follows the sequence count instead, because it holds one token of intermediates for each sequence at once.

The weights and the KV cache divide across the tensor parallel ranks, while the activation buffer and the framework reserve stay in full on every card. Two cards therefore do not halve the footprint. The page reports the fewest cards that hold the run, and then the smallest card that reaches that count.