What is the best LLM for 32 GB of VRAM?

32 GB runs a 30-32B model at Q6-Q8, near-lossless. Qwen3-Coder-30B at Q8 (about 31 GB) and Qwen3 32B at Q6 (about 27 GB) are top picks, and KAT-Coder-34B fits at Q6_K_L (about 29 GB). A 70B becomes worthwhile here at roughly 3-bit (DeepSeek-R1-Distill 70B at IQ3_M, about 32 GB).

What LLM can an RTX 5090 run?

The RTX 5090's 32 GB runs 30-32B models at Q6-Q8 entirely on the card, including Qwen3-Coder-30B at Q8 and Qwen3 32B at Q6. It also runs a 70B such as DeepSeek-R1-Distill-Llama-70B at about 3-bit. For larger models you need a 128 GB unified-memory machine instead.

Is 32 GB of VRAM enough for a 70B model?

At roughly 3-bit, yes. A 70B model like DeepSeek-R1-Distill-Llama-70B fits 32 GB at IQ3_M (about 32 GB), and at 3-bit a 70B is genuinely competitive. For a 70B at 4-bit or higher you need more memory, such as a 128 GB unified-memory system.

Best LLM for 32 GB of VRAM

By Lefi Abdelmonem

Author · AI Local Check

32 GB of VRAM — the RTX 5090's headline number — is the top of the consumer ladder. It runs a 30–32B model at Q8 (near-lossless), or a 70B at 3-bit, all on a single card. If you want the most capable local models without moving to a workstation or unified-memory box, this is the tier. Every figure is measured from real GGUF files.

The best models for 32 GB of VRAM

Model	Size	Best quant that fits	Memory
Qwen3-Coder 30B	30.5B MoE	Q8_0	≈ 31.4 GB
Qwen2.5-Coder 32B	32.8B	Q3_K_M	≈ 31.4 GB
KAT-Coder 34B	34.7B MoE	Q6_K_L	≈ 29.1 GB
Qwen3 32B	32.8B	Q6_K	≈ 26.8 GB
DeepSeek-R1-Distill 70B	70.6B	IQ3_M	≈ 31.8 GB

Memory is weights plus KV cache and system margin at a 4,096-token context.

What to pick

32 GB buys you quality headroom. The same 30–35B models that run at Q5 on a 24 GB card now run at Q6–Q8 here, which is the difference between "very good" and "essentially lossless":

Qwen3-Coder-30B at Q8 (≈ 31 GB) — the fast MoE coder at maximum fidelity.
Qwen3 32B at Q6 (≈ 27 GB) — a top general model with room to spare for a long context.
KAT-Coder-34B at Q6_K_L (≈ 29 GB) — agentic coding at high quality.

You can also push into 70B territory at ~3-bit (DeepSeek-R1-Distill 70B, IQ3_M ≈ 32 GB). At 3-bit a 70B becomes genuinely competitive — this is the first tier where a 70B is worth running over a well-quantized 32B.

32 GB vs a 128 GB unified-memory machine

32 GB is the ceiling for single-GPU consumer cards. To go bigger — a 70B at Q4+, or a 100B+ MoE — you move to a different class of hardware: a 128 GB unified-memory machine (Apple Silicon, AMD Strix Halo) that trades raw GPU speed for capacity. Those run models like Tencent's Hy3 or Inkling-Small that a 32 GB card can't hold.

Tips for 32 GB

Run your 30–32B at Q6–Q8 — you have the room, so take the quality.
A 3-bit 70B is finally worth it here, unlike on 24 GB.
Long context is comfortable — efficient-attention 30B models leave plenty of margin.

Size any model to your card with the calculator, read the RTX 5090 guide, or compare the 24 GB tier.

→ Check what your exact GPU can run