Can you run KAT-Coder-V2.5-Dev locally?

Yes. KAT-Coder-V2.5-Dev is a 35 billion parameter Mixture-of-Experts model released under Apache 2.0, and it runs entirely on a single 24 GB consumer GPU. Computed from the real GGUF files at a 32k context, Q4_K_M needs about 21.35 GB and IQ4_XS about 18.94 GB. A 16 GB card can run it at Q3_K_S or a smaller quantization.

Can a 24 GB GPU run KAT-Coder at its full 262k context?

Yes, at IQ4_XS. The model uses hybrid attention: only 10 of its 40 layers use full attention, so the KV cache costs roughly 20 KiB per token. The full 262,144-token context needs about 5 GB of KV cache, bringing IQ4_XS to about 23.31 GB, which fits a 24 GB card. At Q4_K_M the full context reaches 25.72 GB and needs a little offloading, but 131,072 tokens fit at 23.22 GB.

Is KAT-Coder better than Qwen3.6-35B-A3B?

On the benchmark table Kwaipilot publishes on its own model card, KAT-Coder-V2.5-Dev scores 69.40 on SWE-bench Verified against 64.40 for the Qwen3.6-35B-A3B base it is post-trained from, and 35.80 for Qwen3-Coder-30B. These are vendor-published figures, but the comparison against its own base model at identical size is the meaningful one.

Does KAT-Coder support images?

No. Although the configuration file contains a vision section and Hugging Face tags the model as image-text-to-text, the open-weight release ships the language model only. Kwaipilot's model card states there is no vision tower, and vLLM requires a --language-model-only flag to load it. Treat it as a text model.

Run KAT-Coder Locally: 69.4 SWE-bench on a 24 GB GPU

Q: How much VRAM does KAT-Coder need?

At a 32k context, measured from the published GGUF files: Q2_K about 13.18 GB, Q3_K_M about 16.54 GB, IQ4_XS about 18.94 GB, Q4_K_M about 21.35 GB, Q5_K_S about 23.92 GB, Q6_K about 29.41 GB and Q8_0 about 35.80 GB, including KV cache and system overhead.

By Lefi Abdelmonem

Author · AI Local Check · Published July 28, 2026

Most of the big open-weight news this month has ended the same way: the model is magnificent, and it needs a server rack. Kimi K3 wants 350 GB at its very smallest. Inkling needs 226 GB. DeepSeek V4, 146 GB.

KAT-Coder-V2.5-Dev is the opposite story. Released on July 23, 2026 by Kwaipilot (Kuaishou) under a plain Apache 2.0 licence, it's a 35B Mixture-of-Experts coding agent that fits comfortably on a single 24 GB graphics card — and because of how its attention is built, its full 262,144-token context costs almost nothing.

Why this one matters

It's post-trained on Qwen3.6-35B-A3B, and on the benchmark table Kwaipilot publishes on its own model card it scores 69.40 on SWE-bench Verified — against 64.40 for the Qwen3.6-35B-A3B base it was built from, and 35.80 for Qwen3-Coder-30B. Treat vendor benchmarks with the usual caution, but the comparison against its own base model is the meaningful one: the post-training bought a real gain at identical size.

The demand is visibly local. The official BF16 repository has around 5,300 downloads; bartowski's GGUF conversion alone has over 28,000 — more than five times as many. People are not benchmarking this one in the cloud, they're pulling it onto their own machines.

Key facts
• 34.66 billion parameters total, MoE: 256 experts, 8 active per token + 1 shared (~3B active)
• Licence: Apache 2.0 · Released: July 23, 2026
• Native context: 262,144 tokens
• Text only — the open release ships the language model without the vision tower

How much VRAM does KAT-Coder need?

Computed from the real GGUF file sizes, at a 32k context:

Quantization	Weights	Total (32k ctx)	Runs on
Q2_K	11.75 GB	13.18 GB	16 GB card (low quality)
Q3_K_M	15.11 GB	16.54 GB	20 GB / tight on 16 GB
IQ4_XS	17.51 GB	18.94 GB	24 GB, lots of headroom
Q4_K_M	19.92 GB	21.35 GB	24 GB
Q5_K_S	22.50 GB	23.92 GB	24 GB (just fits)
Q6_K	27.99 GB	29.41 GB	32 GB card
Q8_0	34.38 GB	35.80 GB	48 GB / offload

A 24 GB card — RTX 3090, 4090 or 5090 — runs it at Q4_K_M or Q5_K_S entirely on the GPU. A 16 GB card gets there at Q3_K_S or below.

The part nobody mentions: context is almost free

This is what makes KAT-Coder unusually well suited to agentic coding, where long context is the whole game.

Its 40 layers are hybrid: only 10 of them use full attention, the other 30 use linear attention whose state doesn't grow with the conversation. With just 2 key-value heads on those 10 layers, the KV cache costs roughly 20 KiB per token — an order of magnitude less than a conventional model of this size.

What that means in practice, on a 24 GB GPU:

Context	KV cache	Q4_K_M total	IQ4_XS total
4,096	0.08 GB	20.80 GB ✅	18.39 GB ✅
32,768	0.62 GB	21.35 GB ✅	18.94 GB ✅
131,072	2.50 GB	23.22 GB ✅	20.81 GB ✅
262,144 (full)	5.00 GB	25.72 GB — offload	23.31 GB ✅

Going from a 4k conversation to 131k costs only 2.4 GB of extra VRAM. And at IQ4_XS, the entire 262k context fits in a 24 GB card with room to spare. On a conventional 35B model, a 262k context alone would swallow far more memory than the weights.

A note on "3B active parameters": the MoE design activates only ~3B parameters per token, which makes it fast — but it does not reduce VRAM. All 256 experts must be resident in memory (or streamed from system RAM). Active-parameter counts buy speed, not footprint.

One thing to know before you download

The model's configuration file still contains a vision section and Hugging Face tags it as image-text-to-text, but the open release ships the language model only — Kwaipilot's own model card states there is no vision tower, and vLLM needs a --language-model-only flag. Treat it as a text model.

Check it against your own GPU

See the live figures for every quantization on the KAT-Coder-V2.5-Dev model page, adjust the context to match how you actually work, or pick your graphics card to see everything that fits it. If the quantization labels are new to you, start with GGUF quantization explained.

The bottom line

KAT-Coder-V2.5-Dev is the rare 2026 release that is both genuinely strong and genuinely runnable: Apache 2.0, ~20 GB at Q4_K_M, and a hybrid-attention design that lets a single 24 GB consumer GPU hold a 262k-token context. If you've been waiting for a local coding agent that doesn't need a server, this is the one to try. Check exactly what your PC can run.

→ Check what your exact GPU can run