Best LLM for 8 GB of VRAM
Author · AI Local Check
An 8 GB graphics card — an RTX 3050, 4060, or any 8 GB laptop GPU — is the most common setup for people starting with local AI. The good news: 8 GB is enough to run a genuinely useful model entirely on the GPU. The trick is staying in the 7–8B range and choosing the right quantization. Every figure below is measured from real GGUF files.
The best models for 8 GB of VRAM
| Model | Size | Best quant that fits | Memory |
|---|---|---|---|
| Llama 3.1 8B | 8B | Q6_K_L | ≈ 7.7 GB |
| Qwen3 8B | 8.2B | Q6_K | ≈ 7.6 GB |
| Qwen2.5 7B | 7.6B | Q6_K_L | ≈ 7.1 GB |
| Qwen2.5-Coder 7B | 7.6B | Q5_0 | ≈ 6.0 GB |
| Qwen3 4B | 4B | Q8_0 | ≈ 5.4 GB |
Memory is weights plus KV cache and system margin at a 4,096-token context.
What to pick
For general use, an 8B model at Q6 is the best 8 GB can do while staying fully on the GPU — Llama 3.1 8B or Qwen3 8B are both excellent. If you want a bit more headroom for a longer context, drop to Q5_K_M and you'll have a comfortable 1–2 GB to spare. For coding, Qwen2.5-Coder-7B fits easily at Q5.
Want to run a 4B model at full quality? Qwen3 4B at Q8_0 (≈ 5.4 GB) keeps almost all of the model's quality and leaves plenty of room for context — a great choice if speed and long conversations matter more than raw size.
Can 8 GB run a 13B or bigger?
Only by offloading part of the model to system RAM, which works but is noticeably slower. A 14B model needs roughly 11–12 GB at a usable quantization, so it spills past 8 GB. If you regularly want 13B+, a 12 GB card is the next step up.
Tips for 8 GB
- Leave a margin. Your OS and display already use some VRAM, so target ~7 GB of model, not the full 8.
- Shorter context saves memory. Dropping from a very long context to 4K frees up room for a higher quantization.
- Q4_K_M is the floor for quality. Below Q4 you start to feel the loss; at 8 GB you rarely need to go there for a 7–8B model.
To see exact numbers for any model on your card, run it through the calculator, or read GGUF quantization explained.