Best Local AI Models for NVIDIA RTX 3080 (10 GB VRAM)

Q: What is the VRAM and memory bandwidth of the NVIDIA RTX 3080?

The NVIDIA RTX 3080 features 10 GB of VRAM and a memory bandwidth of 760 GB/s (GDDR6X 320-bit). In local language model inference, VRAM determines which model sizes fit on the card, while memory bandwidth dictates how many tokens per second the GPU generates.

Q: What is the best local AI model to run on a NVIDIA RTX 3080?

The best overall model for the NVIDIA RTX 3080 is Qwen/Qwen3-8B-GGUF using the recommended Q8_0 quantization (9.47 GB total memory). With 10 GB of VRAM, this GPU typically runs a 7–8B model at Q6, entirely in VRAM at full GPU speed. Check our guide to the best LLMs for 8 GB VRAM for details.

The NVIDIA RTX 3080 has 10 GB of VRAM. Here are the popular AI models it can run locally (4,096-token context, ~16.0 GB system RAM assumed), ranked by popularity.

Verdict: High bandwidth 10 GB card: very fast for 8B models, though constrained by 10 GB capacity for 14B.

VRAM & Memory

10 GB

GDDR6X 320-bit

Bandwidth

760 GB/s

Ampere

Fits in VRAM

29 models

Zero offload

TDP / Assumed RAM

320W

RAM: 16.0 GB

At a glance — 29 of the 36 most popular models run fully in VRAM on the NVIDIA RTX 3080 (10 GB), with about 127 tokens/s estimated on an 8B model in Q4. A solid pick is Qwen/Qwen3-8B-GGUF (Q8_0, 9.47 GB, ~93.7 t/s estimated).

The NVIDIA RTX 3080 is built on NVIDIA's Ampere architecture featuring GDDR6X 320-bit delivering 760 GB/s of raw memory bandwidth. High bandwidth 10 GB card: very fast for 8B models, though constrained by 10 GB capacity for 14B.

Equipped with 10 GB of dedicated VRAM, the NVIDIA RTX 3080 can run 29 popular open-source models completely in GPU memory without offloading. This includes full-speed execution for weights like Qwen3-8B-GGUF, Qwen3-4B-GGUF, Qwen2.5-7B-Instruct-GGUF. For a comprehensive breakdown of compatible model weights, see our guide to the best LLMs for 8 GB VRAM.

With a memory bandwidth of 760 GB/s, this card can generate tokens at an estimated peak rate of ~126.7 tokens/second on an 8B parameter model (Q4_K_M). Its rated power draw is 320W TDP, so ensure your system's power supply and case ventilation are adequate for sustained local inferencing.

New to this? Read: How much VRAM do you need?

How the NVIDIA RTX 3080 compares with similar cards

Compared with the NVIDIA RTX 2080 Ti, the NVIDIA RTX 3080 has 1 GB less VRAM (10 vs 11 GB), so fewer models fit fully on the GPU; about 23% faster token generation (~127 vs ~103 tokens/s on an 8B model in Q4, from 760 vs 616 GB/s); a higher board power (320 vs 250 W) and a newer architecture (Ampere vs Turing). Side-by-side comparison
Compared with the NVIDIA RTX 5070, the NVIDIA RTX 3080 has 2 GB less VRAM (10 vs 12 GB), so fewer models fit fully on the GPU; about 13% faster token generation (~127 vs ~112 tokens/s on an 8B model in Q4, from 760 vs 672 GB/s); a higher board power (320 vs 250 W) and an older architecture (Ampere vs Blackwell). Side-by-side comparison
Compared with the NVIDIA RTX 3070 Ti, the NVIDIA RTX 3080 has 2 GB more VRAM (10 vs 8 GB), so larger models or longer contexts fit; about 25% faster token generation (~127 vs ~101 tokens/s on an 8B model in Q4, from 760 vs 608 GB/s) and a higher board power (320 vs 290 W). Side-by-side comparison

Token speeds here are a conservative estimate (memory bandwidth ÷ 6, for an 8B model in Q4 that reads about 5 GB of weights per token, plus overhead); prompt processing depends on GPU compute instead.

29 fit fully in VRAM · 7 run with offload

Only show models that fit fully in VRAM

Model	Size	Quant.	Quality	Memory	Speed~	Verdict
Qwen/Qwen3-8B-GGUF	8.19B	Q8_0	Excellent	9.47 GB	93.7 t/s	Fits in VRAM
Qwen/Qwen3-4B-GGUF	4.02B	Q8_0	Excellent	5.35 GB	190.6 t/s	Fits in VRAM
Qwen/Qwen2.5-7B-Instruct-GGUF	7.62B	Q8_0	Excellent	8.56 GB	100.8 t/s	Fits in VRAM
unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF	8.0B	Q8_K_XL	Excellent	8.48 GB	99.6 t/s	Fits in VRAM
Qwen/Qwen2.5-1.5B-Instruct-GGUF	1.54B	GGUF	Excellent	4.23 GB	229.2 t/s	Fits in VRAM
unsloth/LFM2.5-24B-A2B-GGUF	23.97B	IQ3_XXS	Low	9.22 GB	91.1 t/s	Fits in VRAM
unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF	30.53B	IQ1_S	Very low	9.48 GB	91.5 t/s	Fits in VRAM
Qwen/Qwen2.5-0.5B-Instruct-GGUF	0.49B	GGUF	Excellent	2.03 GB	644.4 t/s	Fits in VRAM
unsloth/Ornith-1.0-9B-GGUF	—	Q8_0	Excellent	9.8 GB	85.7 t/s	Fits in VRAM
Qwen/Qwen3-14B-GGUF	14.77B	Q4_K_M	Good	9.81 GB	90.7 t/s	Fits in VRAM
Qwen/Qwen2.5-Coder-7B-Instruct-GGUF	7.62B	Q8_0	Excellent	8.56 GB	100.8 t/s	Fits in VRAM
ggml-org/gpt-oss-120b-GGUF	116.83B	BF16	Very low	2.42 GB	514.3 t/s	Fits in VRAM
LiquidAI/LFM3-1.7B-GGUF	1.73B	BF16	Excellent	4.08 GB	235.4 t/s	Fits in VRAM
janhq/Jan-v3.5-4B-gguf	4.41B	GGUF	Excellent	9.59 GB	92.4 t/s	Fits in VRAM
Qwen/Qwen2.5-3B-Instruct-GGUF	3.09B	GGUF	Excellent	7.27 GB	120.0 t/s	Fits in VRAM
unsloth/GLM-4.7-Flash-GGUF	31.22B	IQ1_S	Very low	9.62 GB	88.2 t/s	Fits in VRAM
bartowski/Meta-Llama-3.1-8B-Instruct-GGUF	8.03B	Q8_0	Excellent	9.23 GB	95.5 t/s	Fits in VRAM
unsloth/Llama-3.2-3B-Instruct-GGUF	3.21B	F16	Excellent	7.23 GB	126.8 t/s	Fits in VRAM
LiquidAI/LFM2.5-1.2B-Instruct-GGUF	1.17B	BF16	Excellent	3.03 GB	348.2 t/s	Fits in VRAM
lmstudio-community/gemma-3-1b-it-GGUF	1.0B	Q8_0	Excellent	1.96 GB	763.2 t/s	Fits in VRAM
bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF	34.66B	IQ2_XXS	Very low	9.99 GB	83.5 t/s	Fits in VRAM
Qwen/Qwen3-0.6B-GGUF	0.75B	Q8_0	Excellent	1.83 GB	1276.2 t/s	Fits in VRAM
bartowski/Qwen_Qwen3-0.6B-GGUF	0.75B	BF16	Excellent	2.64 GB	540.7 t/s	Fits in VRAM
unsloth/Qwen3-1.7B-GGUF	1.72B	BF16	Excellent	4.45 GB	236.7 t/s	Fits in VRAM
LiquidAI/LFM2.5-350M-GGUF	0.35B	BF16	Excellent	1.51 GB	1147.0 t/s	Fits in VRAM
Qwen/Qwen2.5-Coder-14B-Instruct-GGUF	14.77B	Q4_K_M	Good	9.92 GB	90.8 t/s	Fits in VRAM
unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF	8.19B	Q8_0	Excellent	9.47 GB	93.7 t/s	Fits in VRAM
Qwen/Qwen2.5-Coder-1.5B-Instruct-GGUF	1.54B	Q8_0	Excellent	2.67 GB	430.7 t/s	Fits in VRAM
bartowski/Qwen2.5-14B-Instruct-GGUF	14.77B	Q4_K_M	Good	9.92 GB	90.8 t/s	Fits in VRAM
unsloth/gpt-oss-20b-GGUF	20.91B	F16	Very good	13.74 GB	3.9 t/s	Offload
bartowski/Qwen2.5-32B-Instruct-GGUF	32.76B	Q5_K_L	Very good	23.91 GB	2.3 t/s	Offload
unsloth/flint-GGUF	39.79B	Q4_K_XL	Very good	24.24 GB	2.1 t/s	Offload
ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF	31.58B	Q4_0	Good	18.42 GB	2.8 t/s	Offload
bartowski/Llama-3.3-70B-Instruct-GGUF	70.55B	IQ2_M	Low	24.68 GB	2.2 t/s	Offload
unsloth/Qwen3-Coder-Next-GGUF	79.67B	IQ2_M	Very low	24.42 GB	2.2 t/s	Offload
bartowski/Nanbeige_Nanbeige4.3-33B-GGUF	33.12B	Q5_K_M	Very good	24.96 GB	2.1 t/s	Offload

"Fits in VRAM" = fast, fully on GPU. "Offload" = part on system RAM, slower. Speed is a rough estimate from the NVIDIA RTX 3080's 760 GB/s memory bandwidth (offloaded models are limited by system RAM).

Frequently asked questions

What is the VRAM and memory bandwidth of the NVIDIA RTX 3080?

The NVIDIA RTX 3080 features 10 GB of VRAM and a memory bandwidth of 760 GB/s (GDDR6X 320-bit). In local language model inference, VRAM determines which model sizes fit on the card, while memory bandwidth dictates how many tokens per second the GPU generates.

What is the best local AI model to run on a NVIDIA RTX 3080?

The best overall model for the NVIDIA RTX 3080 is Qwen/Qwen3-8B-GGUF using the recommended Q8_0 quantization (9.47 GB total memory). With 10 GB of VRAM, this GPU typically runs a 7–8B model at Q6, entirely in VRAM at full GPU speed. Check our guide to the best LLMs for 8 GB VRAM for details.

What models can you run on an 8 GB GPU like the NVIDIA RTX 3080?

With 8 GB of VRAM, the NVIDIA RTX 3080 comfortably runs 7B and 8B models (such as Llama 3.1 8B, Mistral 7B, or Qwen 2.5 7B) using Q4_K_M or Q5_K_M quantizations (~5.5 to 6.8 GB). Running 14B models requires offloading memory to system RAM, which lowers token generation speed.

What power supply (PSU) do you need for the NVIDIA RTX 3080 when running local AI?

The NVIDIA RTX 3080 has a rated TDP of 320W. While local inference typically draws less power than full 3D rasterization or gaming, continuous token generation can sustain high loads. A quality power supply of at least 620W is strongly recommended.