Measured runs on one desktop, checked token for token against the same model running plainly. If a number is on this page, there is a run behind it.
| Model | Size, full precision | Full precision | Q8 | Q4 | Q2 |
|---|---|---|---|---|---|
| Llama 3.2 3B | 6.4 GB | 0.051exact | 0.02599.7% agree | 0.02397.5% agree | 0.02482.2%, often differs |
| Llama 3.1 8B | 16 GB | 0.255exact | 0.04599.4% agree | 0.04196.9% agree | 0.04289.7% agree |
| 13B (merge) | 26 GB | 0.348exact | 0.07799.4% agree | 0.06894.7% agree | 0.07183.1%, often differs |
| Mistral Small 3 24B | 47 GB | 1.364exact | 0.501100% agree | 0.11396.6% agree | 0.10789.7% agree |
| Llama 3.1 70B | 141 GB | 1.700exact | 1.105100% agree | 0.86098.1% agree | 0.88493.1% agree |
Seconds per token, lower is faster. Prompt: "Explain how a refrigerator works, step by step." 64 tokens, median of 3 runs, every run checked against the same model running plainly. Agreement measured across 320 positions in 5 prompts. Full precision build c874286, smaller sizes f345571. Lab results, October 4 to 6, 2026. Achieved using capabilities from our other projects; not part of this campaign.
Short common words are one token; longer words split into two or three. Measured on our demo answers: 64 tokens came to 48 words, three quarters of a word each.
Seconds to write one token. Lower is faster. 0.050 is about 20 tokens a second; 1.700 is about 35 tokens a minute.
Exact: the same answer as the plain model, word for word. Agree: how often a smaller size picks the same next word as full precision.
Every full-precision run is compared with a plain run of the same model, token by token. All 64 tokens have to match. The smaller sizes are scored for agreement with full precision, across 320 positions in 5 different prompts.
These figures state what the result is, not how it is achieved. Achieved using capabilities from our other projects; not part of this campaign.
