Tool · Memory and speed
LLM VRAM calculator: how much memory does a model need?
A model needs roughly its parameter count × bits per weight ÷ 8, plus ~1.5 GB of runtime and ~5% for context. At 4-bit (Q4_K_M) that's ~7 GB for Qwen3.5 9B, ~13 GB for GPT-OSS 20B and ~46 GB for Llama 3.3 70B. Pick a model below to see which rigs fit it.
Calculate memory and speed for any model
Needs about ~46 GB (weights 42.4 GB + runtime 1.5 GB + context (KV cache) 2.1 GB)
| Tracked rig | Price | Usable memory | Fit and est. speed |
|---|---|---|---|
| Mac mini M5 Pro | $1,699 | ~17 GB | Won't fit |
| GMKtec EVO-X2 64GB | $2,200 | ~48 GB | Tight fit~5 tok/s |
| Mac Studio M5 Max 36GB | $2,499 | ~27 GB | Won't fit |
| Mac Studio M5 Max 48GB | $3,099 | ~36 GB | Won't fit |
| GMKtec EVO-X2 128GB | $3,500 | ~96 GB | Fits~5 tok/s |
| Minisforum MS-S1 Max | $3,919 | ~96 GB | Fits~5 tok/s |
| Beelink GTR9 Pro | $4,349 | ~96 GB | Fits~5 tok/s |
| Framework Desktop DIY | $4,839 | ~96 GB | Fits~5 tok/s |
| Gigabyte GeForce RTX 4090 Windforce V2card only | $4,899 | ~22 GB | Won't fit |
| Mac Studio M5 Ultra 96GB | $5,499 | ~72 GB | Fits~23 tok/s |
| Lenovo ThinkStation PGX | $5,859 | ~109 GB | Fits~5 tok/s |
| ASUS Ascent GX10 | $5,999 | ~109 GB | Fits~5 tok/s |
| NVIDIA DGX Spark | $6,950 | ~109 GB | Fits~5 tok/s |
| HP ZGX Nano G1n | $7,699 | ~109 GB | Fits~5 tok/s |
| NVIDIA GeForce RTX 5090 Founders Editioncard only | $8,500 | ~30 GB | Won't fit |
| PNY NVIDIA RTX PRO 6000 Blackwellcard only | $17,899 | ~89 GB | Fits~34 tok/s |
| Gigabyte AI TOP ATOM | — | ~109 GB | Fits~5 tok/s |
| Mac Studio M5 Ultra 256GB | — | ~192 GB | Fits~23 tok/s |
Context above ~8K is a rough linear scaling of our 8K allowance; real KV-cache size depends on the model's attention design (grouped-query and sliding-window attention use much less). Speeds are single-user decode estimates. How we estimate
How much memory does each popular model need?
| Model | 4-bit | 8-bit | FP16 |
|---|---|---|---|
| Qwen3.5 9Bdense · 9B | ~7 GBQ4_K_M | ~12 GB | ~20 GB |
| Gemma 4 12Bdense · 12B | ~9 GBQ4_K_M | ~15 GB | ~27 GB |
| GPT-OSS 20BMoE · 21B · 3.6B active | ~13 GBMXFP4 (native) | ~25 GB | ~46 GB |
| Gemma 4 26B-A4BMoE · 25.2B · 3.8B active | ~18 GBQ4_K_M | ~30 GB | ~54 GB |
| Qwen3.8 27Bdense · 27B | ~19 GBQ4_K_M | ~32 GB | ~58 GB |
| Gemma 3 27Bdense · 27.4B | ~19 GBQ4_K_M | ~32 GB | ~59 GB |
| GLM-4.7-Flash 30B-A3BMoE · 30B · 3B active | ~21 GBQ4_K_M | ~35 GB | ~64 GB |
| Qwen3 30B-A3BMoE · 30.5B · 3.3B active | ~21 GBQ4_K_M | ~36 GB | ~66 GB |
| Gemma 4 31Bdense · 31B | ~21 GBQ4_K_M | ~36 GB | ~67 GB |
| Qwen3 32Bdense · 32.8B | ~22 GBQ4_K_M | ~38 GB | ~70 GB |
| DeepSeek R1 Distill Qwen 32Bdense · 32.8B | ~22 GBQ4_K_M | ~38 GB | ~70 GB |
| Qwen3.6 35B-A3BMoE · 35B · 3B active | ~24 GBQ4_K_M | ~41 GB | ~75 GB |
| Llama 3.3 70Bdense · 70B | ~46 GBQ4_K_M | ~80 GB | ~148 GB |
| DeepSeek R1 Distill Llama 70Bdense · 70.6B | ~46 GBQ4_K_M | ~80 GB | ~150 GB |
| Qwen3-Next 80B-A3BMoE · 80B · 3B active | ~52 GBQ4_K_M | ~91 GB | ~170 GB |
| Llama 4 Scout 109B-A17BMoE · 109B · 17B active | ~71 GBQ4_K_M | ~123 GB | ~230 GB |
| GPT-OSS 120BMoE · 117B · 5.1B active | ~67 GBMXFP4 (native) | ~132 GB | ~247 GB |
| Mistral Small 4 119B-A6BMoE · 119.4B · 6.5B active | ~78 GBQ4_K_M | ~135 GB | ~252 GB |
| Nemotron 3 Super 120B-A12BMoE · 120B · 12B active | ~78 GBQ4_K_M | ~135 GB | ~254 GB |
| Qwen3.5 122B-A10BMoE · 122B · 10B active | ~79 GBQ4_K_M | ~138 GB | ~258 GB |
| Qwen3 235B-A22BMoE · 235B · 22B active | ~151 GBQ4_K_M | ~264 GB | ~495 GB |
| Muse Spark Open 816B-A42BMoE · 816B · 42B active | ~521 GBQ4_K_M | ~912 GB | ~1,715 GB |
Weights + runtime + ~8K context, from the formula below.
How is LLM memory calculated?
memory ≈ parameters × bits per weight ÷ 8 + 1.5 GB + 5% of weights for ~8K context
Speed follows memory bandwidth: every generated token reads the active weights once, so tok/s ≈ 1 ÷ (active GB ÷ (efficiency × bandwidth) + overhead), with efficiency and per-token overhead fitted per platform to 52 public benchmarks.
Frequently asked questions
How do you calculate VRAM for an LLM?
Multiply the parameter count (in billions) by the bits per weight and divide by 8 to get GB of weights, then add ~1.5 GB for the runtime and ~5% of the weights for ~8K tokens of context. Q4_K_M averages ~4.85 bits per weight and Q8_0 ~8.5.
How much VRAM do I need for a 70B model?
About 46 GB at 4-bit and 80 GB at 8-bit for Llama 3.3 70B, so it won't fit on a single 24 GB or 32 GB card. The cheapest tracked rig that holds it is the GMKtec EVO-X2 64GB at $2,200.
How much VRAM do I need for a 7B–9B model?
About 7 GB at 4-bit for Qwen3.5 9B, so an 8 GB card is tight and 12 GB is comfortable. At 8-bit it needs ~12 GB.
Does a longer context need more memory?
Yes. The KV cache grows with every token of context. Our default allowance covers ~8K tokens; the calculator scales it linearly for 32K and 128K, which is a rough upper-end guess because many newer models use attention designs that need far less.
How accurate are the speed numbers?
They come from a bandwidth model fitted to 43 public llama.cpp/MLX runs and checked on 9 held-out runs. Treat them as estimates; prompt processing, batching and speculative decoding aren't modelled.