Methodology

How we measure: LLM fit, tokens/sec and prices

We estimate memory as weights plus runtime overhead plus ~8K tokens of context, and decode speed from memory bandwidth divided by the weights read per token, with per-platform constants fitted to 43 public benchmarks (median error 5%). Prices are copied from retailer pages by script, with the date checked.

By Best Local AI Rig Team · Updated 2026-10-24 · Prices & stock checked 2026-10-24 (prices change daily, so confirm on the store page) · How we measure

Where do prices and stock come from?

A script checks each retailer page and records the price and stock text exactly as shown, with the date. If we couldn't see a price, we say “unknown” instead of guessing. GPU prices are for the card only. Prices exclude tax and shipping. Always confirm on the retailer's page before buying.

How do we decide if a model will fit?

memory needed (GB) = weights + 1.5 GB + 5% of weights
weights = total parameters (billions) × bits per weight ÷ 8

• Q4_K_M: 4.85 bits per weight. ~4.85 bits/weight average for llama.cpp Q4_K_M
• Q8_0: 8.5 bits per weight. ~8.5 bits/weight for llama.cpp Q8_0
• The 1.5 GB + 5% term roughly covers the runtime plus the KV cache at about 8K tokens of context for one user. Long contexts (32K–1M) can need much more; models with sliding-window or compressed attention need less.

Usable memory is installed memory times a per-machine fraction. Discrete GPUs get about 93%, since the driver and display take the rest. DGX Spark/GB10 boxes get about 85%, because memory is shared with the OS. Macs and Strix Halo get about 75%: macOS and the AMD driver cap GPU-usable memory by default. You can raise that cap with tuning, so treat “Won't fit” near the line as “maybe”.

A model gets Fits under 90% of usable memory, Tight fit between 90% and 100%, and Won't fit above that. We don't count offloading to system RAM, which works but is much slower.

How do we estimate tokens per second?

seconds per token ≈ active weights (GB) ÷ (efficiency × memory bandwidth (GB/s)) + fixed overhead
active weights (GB) = active parameters (B) × bits per weight ÷ 8
tokens/sec = 1 ÷ seconds per token

Generating each token means reading every active weight from memory once, so bandwidth sets the ceiling. For mixture-of-experts (MoE) models, only the active parameters count. That's why a 120B MoE can be faster than a 70B dense model. On top of the memory reads, every token has a small fixed cost: launching GPU kernels, attention and, for MoE models, routing to the experts. That cost hardly matters for a 70B dense model, but it's most of the time for a small MoE on a fast GPU. That's why gpt-oss-120b runs at about 200 tok/s on an RTX PRO 6000, not the 400 a pure bandwidth formula predicts.

How accurate are the speed estimates?

We fitted the efficiency and overhead per platform to 43 published single-user decode benchmarks (llama.cpp llama-bench tg128/tg32, plus a few MLX runs), covering dense and MoE models on every platform we list:

Against those 43 runs, the median estimate is within 5% of the measured speed, and 37 of 43 land within ±15% (worst case: 24% high, on a Qwen3-Next hybrid-attention model). Our old single-factor formula (0.6 × bandwidth ÷ active weights) had a 26% median error and was up to 2× too optimistic for MoE models on fast GPUs and Macs. The full write-up, with every benchmark and a chart: we checked our speed estimates against 52 real benchmarks.

These are still estimates for one user at short context with llama.cpp-class runtimes. Long contexts are slower (often 20–40% at 32K). New runtime releases, speculative decoding/MTP and batching can be faster, and new architectures (hybrid or linear attention, very large expert counts) can differ. This is decode speed. Prompt processing (prefill) depends on compute, not bandwidth, and that's where NVIDIA GPUs and DGX Spark pull ahead of Macs and Strix Halo.

Benchmark sources: llama.cpp DGX Spark benches, llama.cpp DGX Spark/Strix Halo thread, llama.cpp Apple Silicon thread, llama.cpp CUDA scoreboard, llama.cpp gpt-oss guide, llama.cpp CUDA token-generation thread, llm-tracker Strix Halo, llm-tracker RTX PRO 6000, RTX 5090 benchmark dataset, M5 Max llama.cpp vs MLX, Hardware Corner M5 Max, oMLX M3 Ultra, quantized.uk and a Strix Halo r/LocalLLM write-up.

What do the stock labels mean?

In stock the retailer shows it orderable now. Pre-order pre-order or a stated delay. Out of stock the retailer says out of stock. Stock unknown we couldn't read stock. Hover a label to see the retailer's exact wording.

What sources do we use?

Memory bandwidth comes from manufacturer spec pages. Speed calibration data comes from the public benchmarks linked above. Model parameter counts come from the model makers and public catalogs. Product images come from manufacturer and retailer product pages and press kits, and each source is recorded in our data file. Corrections are welcome: reply to any newsletter or email hello@bestlocalairig.com.

When is the data updated?

• Prices and stock: re-checked by script, most recently 2026-10-24. Each price on the site shows its own check date.
• Speed calibration: fitted October 2026 on 43 runs, with a holdout set; see the benchmark write-up.
• Model list and specs: updated when new open models ship; this page was last rebuilt 2026-10-24.

Who is behind this?

The Best Local AI Rig Team. See About for our editorial and affiliate policy, and email hello@bestlocalairig.com with corrections.