Best Local AI Models for NVIDIA RTX 5090 Laptop (24 GB VRAM)

Q: What is the VRAM and memory bandwidth of the NVIDIA RTX 5090 Laptop?

The NVIDIA RTX 5090 Laptop features 24 GB of VRAM and a memory bandwidth of 896 GB/s (GDDR7 256-bit). In local language model inference, VRAM determines which model sizes fit on the card, while memory bandwidth dictates how many tokens per second the GPU generates.

Q: What is the best local AI model to run on a NVIDIA RTX 5090 Laptop?

The best overall model for the NVIDIA RTX 5090 Laptop is bartowski/Qwen2.5-32B-Instruct-GGUF using the recommended Q5_K_L quantization (23.91 GB total memory). With 24 GB of VRAM, this GPU typically runs a 30–35B model at Q5, fully on the GPU at full GPU speed. Check our guide to the best LLMs for 24 GB VRAM for details.

The NVIDIA RTX 5090 Laptop has 24 GB of VRAM. Here are the popular AI models it can run locally (4,096-token context, ~32.0 GB system RAM assumed), ranked by popularity.

Verdict: With 24 GB of GDDR7 on a 256-bit bus, models up to about 30B fit entirely in VRAM in 4-bit quantization, while 70B models still spill into system RAM. Each laptop maker sets the power limit (95–150 W).

VRAM & Memory

24 GB

GDDR7 256-bit

Bandwidth

896 GB/s

Blackwell

Fits in VRAM

36 models

Zero offload

TDP / Assumed RAM

150W

RAM: 32.0 GB

At a glance — 36 of the 38 most popular models run fully in VRAM on the NVIDIA RTX 5090 Laptop (24 GB), with about 149 tokens/s estimated on an 8B model in Q4. A solid pick is bartowski/Qwen2.5-32B-Instruct-GGUF (Q5_K_L, 23.91 GB, ~40.5 t/s estimated).

The NVIDIA RTX 5090 Laptop is built on NVIDIA's Blackwell architecture featuring GDDR7 256-bit delivering 896 GB/s of raw memory bandwidth. With 24 GB of GDDR7 on a 256-bit bus, models up to about 30B fit entirely in VRAM in 4-bit quantization, while 70B models still spill into system RAM. Each laptop maker sets the power limit (95–150 W).

Equipped with 24 GB of dedicated VRAM, the NVIDIA RTX 5090 Laptop can run 36 popular open-source models completely in GPU memory without offloading. This includes full-speed execution for weights like Qwen2.5-32B-Instruct-GGUF, Qwen3-8B-GGUF, Qwen2.5-7B-Instruct-GGUF. For a comprehensive breakdown of compatible model weights, see our guide to the best LLMs for 24 GB VRAM.

With a memory bandwidth of 896 GB/s, this card can generate tokens at an estimated peak rate of ~149.3 tokens/second on an 8B parameter model (Q4_K_M). Its rated power draw is 150W TDP, so ensure your system's power supply and case ventilation are adequate for sustained local inferencing.

New to this? Read: How much VRAM do you need?

36 fit fully in VRAM · 2 run with offload

Only show models that fit fully in VRAM

Model	Size	Quant.	Quality	Memory	Speed~	Verdict
bartowski/Qwen2.5-32B-Instruct-GGUF	32.76B	Q5_K_L	Very good	23.91 GB	40.5 t/s	Fits in VRAM
Qwen/Qwen3-8B-GGUF	8.19B	Q8_0	Excellent	9.47 GB	110.5 t/s	Fits in VRAM
Qwen/Qwen2.5-7B-Instruct-GGUF	7.62B	GGUF	Excellent	15.21 GB	63.1 t/s	Fits in VRAM
Qwen/Qwen3-4B-GGUF	4.02B	Q8_0	Excellent	5.35 GB	224.8 t/s	Fits in VRAM
unsloth/gpt-oss-20b-GGUF	20.91B	F16	Very good	13.74 GB	69.8 t/s	Fits in VRAM
unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF	30.53B	Q5_K_XL	Very good	21.42 GB	44.3 t/s	Fits in VRAM
LiquidAI/LFM2.5-2.6B-GGUF	2.7B	BF16	Excellent	5.89 GB	178.1 t/s	Fits in VRAM
Qwen/Qwen2.5-1.5B-Instruct-GGUF	1.54B	GGUF	Excellent	4.23 GB	270.2 t/s	Fits in VRAM
Qwen/Qwen2.5-0.5B-Instruct-GGUF	0.49B	GGUF	Excellent	2.03 GB	759.7 t/s	Fits in VRAM
unsloth/Ornith-1.0-9B-GGUF	—	BF16	Excellent	17.61 GB	53.7 t/s	Fits in VRAM
bartowski/Meta-Llama-3.1-8B-Instruct-GGUF	8.03B	Q8_0	Excellent	9.23 GB	112.6 t/s	Fits in VRAM
Qwen/Qwen3-14B-GGUF	14.77B	Q8_0	Excellent	16.05 GB	61.3 t/s	Fits in VRAM
Qwen/Qwen2.5-Coder-7B-Instruct-GGUF	7.62B	GGUF	Excellent	15.21 GB	63.1 t/s	Fits in VRAM
janhq/Jan-v3.5-4B-gguf	4.41B	GGUF	Excellent	9.59 GB	109.0 t/s	Fits in VRAM
bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF	34.66B	Q5_K_S	Very good	23.38 GB	39.8 t/s	Fits in VRAM
LiquidAI/LFM2.5-1.2B-Instruct-GGUF	1.17B	BF16	Excellent	3.03 GB	410.6 t/s	Fits in VRAM
Qwen/Qwen2.5-3B-Instruct-GGUF	3.09B	GGUF	Excellent	7.27 GB	141.5 t/s	Fits in VRAM
ggml-org/gpt-oss-120b-GGUF	116.83B	BF16	Very low	2.42 GB	606.3 t/s	Fits in VRAM
LiquidAI/LFM2.5-230M-GGUF	0.23B	BF16	Excellent	1.28 GB	2082.9 t/s	Fits in VRAM
unsloth/GLM-4.7-Flash-GGUF	31.22B	Q5_K_XL	Very good	21.21 GB	44.3 t/s	Fits in VRAM
unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF	8.0B	BF16	Excellent	14.87 GB	63.9 t/s	Fits in VRAM
unsloth/Qwen3-1.7B-GGUF	1.72B	BF16	Excellent	4.45 GB	279.1 t/s	Fits in VRAM
Qwen/Qwen2.5-Coder-1.5B-Instruct-GGUF	1.54B	Q8_0	Excellent	2.67 GB	507.8 t/s	Fits in VRAM
unsloth/flint-GGUF	39.79B	Q3_K_XL	Fair	18.13 GB	52.0 t/s	Fits in VRAM
unsloth/Qwen-AgentWorld-35B-A3B-GGUF	34.66B	Q4_K_XL	Very good	21.67 GB	43.1 t/s	Fits in VRAM
Qwen/Qwen3-0.6B-GGUF	0.75B	Q8_0	Excellent	1.83 GB	1504.5 t/s	Fits in VRAM
LiquidAI/LFM2.5-24B-A2B-GGUF	23.97B	Q6_K	Excellent	19.22 GB	48.9 t/s	Fits in VRAM
ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF	31.58B	Q4_0	Good	18.42 GB	50.9 t/s	Fits in VRAM
Qwen/Qwen2.5-Coder-14B-Instruct-GGUF	14.77B	Q8_0	Excellent	16.17 GB	61.3 t/s	Fits in VRAM
unsloth/Llama-3.2-3B-Instruct-GGUF	3.21B	F16	Excellent	7.23 GB	149.5 t/s	Fits in VRAM
bartowski/Qwen2.5-14B-Instruct-GGUF	14.77B	Q8_0	Excellent	16.17 GB	61.3 t/s	Fits in VRAM
unsloth/Qwen3-Coder-Next-GGUF	79.67B	IQ2_XXS	Very low	22.89 GB	41.3 t/s	Fits in VRAM
unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF	8.19B	BF16	Excellent	16.63 GB	58.7 t/s	Fits in VRAM
bartowski/Llama-3.2-1B-Instruct-GGUF	1.24B	F16	Excellent	3.3 GB	388.0 t/s	Fits in VRAM
bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF	7.62B	F16	Excellent	15.21 GB	63.1 t/s	Fits in VRAM
LiquidAI/LFM3-1.7B-GGUF	1.73B	BF16	Excellent	4.08 GB	277.5 t/s	Fits in VRAM
unsloth/Ornith-1.5-122B-GGUF	125.09B	IQ3_XXS	Low	47.01 GB	1.1 t/s	Offload
google/gemma-7b	8.54B	GGUF	Excellent	33.11 GB	1.6 t/s	Offload

"Fits in VRAM" = fast, fully on GPU. "Offload" = part on system RAM, slower. Speed is a rough estimate from the NVIDIA RTX 5090 Laptop's 896 GB/s memory bandwidth (offloaded models are limited by system RAM).

Frequently asked questions

What is the VRAM and memory bandwidth of the NVIDIA RTX 5090 Laptop?

The NVIDIA RTX 5090 Laptop features 24 GB of VRAM and a memory bandwidth of 896 GB/s (GDDR7 256-bit). In local language model inference, VRAM determines which model sizes fit on the card, while memory bandwidth dictates how many tokens per second the GPU generates.

What is the best local AI model to run on a NVIDIA RTX 5090 Laptop?

The best overall model for the NVIDIA RTX 5090 Laptop is bartowski/Qwen2.5-32B-Instruct-GGUF using the recommended Q5_K_L quantization (23.91 GB total memory). With 24 GB of VRAM, this GPU typically runs a 30–35B model at Q5, fully on the GPU at full GPU speed. Check our guide to the best LLMs for 24 GB VRAM for details.

Can the NVIDIA RTX 5090 Laptop run 70B parameter models like Llama 3.3 70B?

Yes. With 24 GB of VRAM, the NVIDIA RTX 5090 Laptop can run 70B parameter models quantized to 3-bit or 4-bit (such as IQ3_M or Q4_K_M) with moderate context windows (4k tokens). For larger context windows (16k+), pairing with a second GPU or slight CPU offloading may be required.