RS-Molt is in pre-launch
The campaign is moving to its own home here on r-sym.com, next to the results, the builds and, soon, a live demo. Pledges open at launch. Leave your email on the overview page and you will hear the moment they do.
Campaign news, new builds and, once the demo opens, public runs with their receipts. Newest first.
The campaign is moving to its own home here on r-sym.com, next to the results, the builds and, soon, a live demo. Pledges open at launch. Leave your email on the overview page and you will hear the moment they do.
A kernel fix for the quantized sizes. 70B at Q4 went from 1.313 to 0.860 s/token, and every Q8, Q4 and Q2 cell on every model improved. The answers did not change: all 45 runs produced the same tokens as before.
Llama 3.1 70B, all 141 GB, no quantization, on a desktop with one 16 GB graphics card and 32 GB of memory. Every token matched a plain run of the same model.