Token generation is capped by how fast weights stream from memory. Qwen3.8 Flash-Next is a mixture-of-experts model: only 6B of its 180B parameters are read per token, about 2.8 GB at IQ3_S. Layers are split across mixed cards; the ceiling uses the fastest card as an upper bound: 1493 GB/s ÷ 2.8 GB ≈ 534 tok/s max.
| Model | Quant · ctx | Gen tok/s | PP tok/s | Rank | Status |
|---|---|---|---|---|---|
| Qwen3.8 Flash-Next | IQ3_S · 8k | 99.0 | 1,321 | #1 | Verified run |
Load Qwen3.8 Flash-Next at IQ3_S and run the command. If your rig has the same GPUs and lands within 10%, this result gains a reproduction.
$ npx toks-gg bench