What is the best LLM for 24 GB of VRAM?

24 GB runs a 30-35B model at Q5 entirely on the GPU. Qwen3-Coder-30B (about 21 GB) and KAT-Coder-34B (about 23 GB) are the best for coding, and Qwen3 32B (about 23 GB at Q5_K_M) is a superb general model. A 70B fits only at about 2-bit, which usually feels worse than a 32B at Q5.

Can a 24 GB GPU run a 70B model?

Yes, at roughly 2-bit. DeepSeek-R1-Distill-Llama-70B fits a 24 GB card at IQ2_S (about 23 GB). It runs entirely on the GPU, but 2-bit quantization costs noticeable quality, so a 32B at Q5 often gives better results for everyday use.

Is an RTX 3090 or 4090 good for local LLMs?

Yes. Their 24 GB of VRAM is the enthusiast sweet spot: they run 30-35B models at Q5 fully on the GPU, including fast Mixture-of-Experts coders like Qwen3-Coder-30B and KAT-Coder, with enough headroom for a long context on efficient-attention models.

Best LLM for 24 GB of VRAM

By Lefi Abdelmonem

Author · AI Local Check

24 GB of VRAM — an RTX 3090, 4090 or 5090 — is the enthusiast sweet spot for local AI. It's the tier where a 30B-class model runs entirely on the GPU at a good quantization, and where a 70B first becomes possible at low bits. This is the card most serious local-LLM users aim for. All numbers are measured from real GGUF files.

The best models for 24 GB of VRAM

Model	Size	Best quant that fits	Memory
Qwen3-Coder 30B	30.5B MoE	Q5_K_XL	≈ 21.4 GB
KAT-Coder 34B	34.7B MoE	Q5_K_S	≈ 23.3 GB
Qwen3 32B	32.8B	Q5_K_M	≈ 23.4 GB
Ornith 1.0 35B	34.7B	Q5_K_M	≈ 23.9 GB
DeepSeek-R1-Distill 70B	70.6B	IQ2_S	≈ 22.8 GB

Memory is weights plus KV cache and system margin at a 4,096-token context.

What to pick

For most people the best use of 24 GB is a 30–35B model at Q5, fully in VRAM:

Qwen3-Coder-30B and KAT-Coder-34B are Mixture-of-Experts coders that fit at Q5 (≈ 21–23 GB) and are fast thanks to sparse activation — the standout picks for agentic coding.
Qwen3 32B is a superb dense general model at Q5_K_M (≈ 23 GB).

You can also run a 70B at ~2-bit (DeepSeek-R1-Distill 70B, IQ2_S ≈ 23 GB). It fits, but 2-bit costs real quality — a 32B at Q5 usually feels better than a 70B at 2-bit. Try both and judge for your workload.

The KV-cache advantage

Many of the best 24 GB models use efficient attention (hybrid or grouped-query), so their context is cheap in memory. KAT-Coder, for example, adds only about 2.5 GB of KV cache going from 4K to 131K tokens — so you can run a long context without dropping quantization. That's a real edge for coding and document work.

Tips for 24 GB

A 32B at Q5 is the reliable default — full-quality reasoning, entirely on the GPU.
Prefer a well-quantized 32B over a 2-bit 70B for everyday use.
Exploit cheap context on hybrid/MoE models — long documents cost little extra VRAM.

Check any model against your card with the calculator, or see the RTX 5090 guide and other tiers: 16 GB · 32 GB.

→ Check what your exact GPU can run