How Much RAM Do You Need to Run an LLM?
Author · AI Local Check
Most local-AI guides focus on VRAM — and for GPU inference that's the number that matters. But system RAM matters too, in three specific situations: running on CPU, offloading a model too big for your GPU, and loading very large Mixture-of-Experts models. This page explains how much RAM you actually need, and when.
The short answer
- 16 GB RAM — fine if the model fits entirely in your GPU's VRAM. Enough for CPU-running small models (up to ~3B).
- 32 GB RAM — the comfortable default for local AI. Runs 7–8B models on CPU, and covers modest GPU offloading.
- 64 GB RAM — lets you offload larger models (30B+) from a smaller GPU, or run them on CPU.
- 128 GB RAM (or unified memory) — the tier for very large models: 70B+ on CPU, or 100B+ MoE models on unified-memory machines.
When does RAM matter?
1. Running on CPU (no GPU, or not enough VRAM)
Without a capable GPU, the model runs in system RAM on your CPU. Here RAM plays the role VRAM normally does: you need enough to hold the whole model plus its KV cache. A 7–8B model at Q4 needs roughly 6–8 GB, so 16 GB of RAM handles it; a 14B needs ~12 GB, comfortable on 32 GB. CPU inference is slower than GPU, but perfectly usable for small and mid-size models.
2. Offloading (model bigger than your VRAM)
When a model doesn't fit entirely in VRAM, tools like llama.cpp and Ollama keep the overflow in system RAM. The model still runs, but the layers in RAM are slower. For this you need enough RAM to hold the part that doesn't fit in VRAM, plus headroom for your OS. A practical rule: aim for system RAM at least equal to the model's total size, and ideally more.
3. Large Mixture-of-Experts models
MoE models hold all their experts in memory even though only a few run per token. That makes their total size large — and a common way to run them is to keep the experts in fast system RAM. This is why big MoE models are associated with 64–128 GB machines rather than a single GPU.
A rough RAM guide
| Your RAM | On CPU only | For GPU offloading |
|---|---|---|
| 16 GB | up to ~3–7B | small overflow from an 8–12 GB GPU |
| 32 GB | up to ~14B | comfortable offload for most 24 GB-GPU setups |
| 64 GB | up to ~30B | run a 70B split across GPU + RAM |
| 128 GB | 70B+, or 100B+ MoE | very large models, unified memory |
These are guidance ranges; exact needs depend on the model and quantization.
RAM vs VRAM: which should you upgrade?
If your goal is fast local AI, spend on VRAM first — a model fully on the GPU is far faster than one spilling into RAM. Add system RAM when you want to run models bigger than your GPU can hold, accepting lower speed, or when you run on CPU. In short: VRAM buys speed, RAM buys capacity.
For the GPU side of the equation, see how much VRAM you need. To size a specific model against your machine, use the calculator — it accounts for both VRAM and RAM.