Guides
In-depth, data-backed guides on running AI models locally — VRAM requirements, quantization, hardware choice and more.
How Much VRAM Do You Need to Run an LLM Locally?
How much VRAM and RAM you need to run AI models locally — by model size and GGUF quantization, with real GGUF sizes, calculated estimates and per-GPU guidance.
Read guideCan You Run Ling-3.0-flash Locally?
Ling-3.0-flash is a 127B MoE (MIT) — and at 1-bit it's just 31 GB, fitting a single 32 GB RTX 5090. Measured VRAM per quantization.
Read guideBest Local LLM in 2026 (by GPU & Task)
The best open LLM you can run locally, mapped to your GPU and your task — coding, reasoning, small & fast — with real memory numbers per model.
Read guideBest Local LLM for Reasoning (2026)
The best open reasoning models of 2026 — Qwen3 32B and the DeepSeek-R1 distills — paired with the GPU that runs them. Real memory numbers.
Read guideHow Much RAM Do You Need to Run an LLM?
How much system RAM you need to run AI models locally — for CPU inference, GPU offloading and large MoE models. A clear RAM guide by model size.
Read guideGGUF vs GPTQ vs AWQ: Which Format to Use
GGUF vs GPTQ vs AWQ explained: which quantization format to use for local AI, and why GGUF is the default for llama.cpp, Ollama and LM Studio.
Read guideBest Small LLM (1B–4B) You Can Run Anywhere
The best small language models of 2026 — 0.5B to 4B — that run on a laptop, CPU or any GPU. Real memory numbers, and which tiny model to pick.
Read guidellama.cpp vs Ollama: Engine vs Manager
llama.cpp vs Ollama explained: Ollama is built on llama.cpp. One is the engine, one is the manager — here's which to use, and when.
Read guideBest Local LLM for Coding (2026)
The best open coding models of 2026 paired with the hardware that runs them — from a 7B on 8 GB to a 69.4 SWE-bench coder on a 24 GB GPU.
Read guideBest LLM for 8 GB of VRAM
The best AI models that run on an 8 GB GPU, with real memory numbers per quantization. 7–8B models at Q6 are the sweet spot — here's what to pick.
Read guideBest LLM for 12 GB of VRAM
12 GB runs 14B models at Q5 and a 20B MoE at 4-bit. See the best models for a 12 GB GPU, with memory measured from real GGUF files.
Read guideBest LLM for 16 GB of VRAM
16 GB runs a 14B at high quality or a 24–27B at 4-bit. The best models for a 16 GB GPU, with real memory numbers per quantization.
Read guideBest LLM for 24 GB of VRAM
24 GB (RTX 3090/4090/5090) runs a 30–35B model at Q5 fully on the GPU. The best models for 24 GB VRAM, with real measured memory.
Read guideBest LLM for 32 GB of VRAM
32 GB runs a 30–32B at Q8 or a 70B at 3-bit. The best models for an RTX 5090 or 32 GB GPU, with memory measured from real GGUF files.
Read guideOllama vs LM Studio: Which Should You Use?
Ollama vs LM Studio compared: CLI and API versus a graphical app. Both run the same GGUF models — here's which one fits how you work.
Read guideLocal LLM Glossary: VRAM, GGUF, KV Cache, MoE
Plain-language definitions of the terms behind running AI models locally: VRAM, quantization, GGUF, KV cache, context length, MoE and more.
Read guideRun KAT-Coder Locally: 69.4 SWE-bench on a 24 GB GPU
KAT-Coder-V2.5-Dev is Apache 2.0, scores 69.4 SWE-bench, and fits a 24 GB card — with its full 262k context. Measured VRAM per quant.
Read guideGLM-5.2 Locally: Does REAP Pruning Make a 754B Model Fit?
GLM-5.2 needs ~203 GB even at 1-bit. Does REAP expert pruning make it fit your GPU? Real numbers — and the GLM that does run on 16 GB.
Read guideCan You Run Hy3 (Tencent's 295B Model) Locally?
Hy3 is Tencent's 295B open model. Can you run it locally? At 1-bit it's ~87 GB — small enough for a 128 GB machine. Real numbers inside.
Read guideWhat LLMs Can an RTX 5090 Run Locally?
The RTX 5090's 32 GB VRAM is the new ceiling for local AI. See what it runs — 30B at 8-bit, a 70B at low-bit — with real numbers per quant.
Read guideCan You Run Inkling Locally?
Inkling needs ~226 GB. But Inkling-Small (266B, Apache 2.0) runs in 71–113 GB — it fits a 128 GB machine. Measured numbers for both.
Read guideCan You Run Kimi K3 Locally?
We measured every Kimi K3 build: 1,561 GB native, down to 350 GB after 24h of community compression. Still too big for your PC — here's what fits.
Read guideCan You Run DeepSeek V4 Locally?
DeepSeek V4-Flash-0731 (304B) measured: 78–152 GB by quant. Too big for a gaming GPU, but 1–2 bit fits a 128 GB machine. Plus DeepSeek models that run anywhere.
Read guideCan You Run Qwen 3.6 Locally?
Qwen 3.6 27B is one of 2026's top open-weight models — and yes, you can run it locally. Here's the VRAM needed per GGUF quantization, and which GPU fits.
Read guideCan You Run Claude Sonnet 5 Locally?
Claude Sonnet 5 just launched — but it's cloud-only and can't run locally. Here's why, plus the best open-weight LLMs you can run on your own GPU instead.
Read guideGGUF Quantization Explained (Q4_K_M, Q5, Q8, IQ)
What GGUF quantization levels mean, how they trade memory for quality, and which one to choose for running LLMs locally.
Read guideBest LLM for Your VRAM (8, 12, 16, 24 GB)
Which local LLM to run for your amount of VRAM — the biggest models that fit at a balanced quantization, with examples for each GPU memory size.
Read guideBest GPU for Local LLM Inference (2026)
The best GPU for local LLM inference in 2026, ranked by VRAM — from budget 8 GB cards to the 32 GB RTX 5090, and which models each one runs.
Read guideHow to Run an LLM Locally (Beginner's Guide)
A beginner-friendly guide to running AI models on your own computer — choosing a tool, picking a model that fits, and example commands.
Read guide