Can You Run Nanbeige4.3-33B Locally?

Yes on a 24 GB card, with one catch: context length. Nanbeige4.3-33B is a dense 33.12-billion-parameter model that the Nanbeige team published on Hugging Face on October 24, 2026, under the Apache 2.0 license. Computed from the real GGUF files, the Q4_K_M build needs 20.97 GB at a 4,096-token context, so an RTX 3090 or 4090 runs it fully on the GPU. Stretch the context to 32,768 tokens and the same build needs 26.66 GB, which no longer fits.

The short answer by GPU

Memory is the real size of the GGUF file plus the KV cache at a 4,096-token context and a fixed 0.8 GB system margin. Your desktop and browser also take VRAM, so on a 24 GB card that drives a display, Q4_K_M (20.97 GB) leaves 3.03 GB of room where Q5_K_S leaves almost none. On 12 GB, the IQ2_S file adds up to exactly 12.00 GB, which leaves no headroom beyond the system margin, so the table lists IQ2_XS.

Why context decides the tier

Nanbeige4.3-33B is a plain dense model: the config.json lists a Llama architecture with 52 layers, 56 attention heads and 8 key-value heads of dimension 128, no experts and no sliding window. Every layer keeps a full KV cache, and each token costs 212,992 bytes at 16-bit precision.

At 4,096 tokens that is 0.81 GB. At 32,768 tokens it is 6.50 GB, eight times more, and the jump of 5.69 GB applies to every quantization. This is what moves the model down a tier:

• On 24 GB, a 32K context pushes you from Q5_K_S down to IQ3_M (23.88 GB). Q4_K_M at 32K would need 2.66 GB of offload.
• On 32 GB, a 32K context still fits Q5_K_M (30.65 GB), and Q4_K_M (26.66 GB) leaves comfortable room.
• On 16 GB, nothing fits at 32K: even IQ2_XXS reaches 16.68 GB.

The model card says the model supports up to 524,288 tokens. Our tables stop at 32K because the KV cache keeps growing in a straight line: at the full 524,288 tokens, the cache alone would take 104 GB. Quantizing the KV cache in llama.cpp lowers this cost; the figures here assume the default 16-bit cache.

Memory per quantization

We read the file list of bartowski's GGUF repository on November 2, 2026: 25 quantizations from IQ2_XXS to BF16, 109,111 downloads, last modified October 25. The Nanbeige team has not published its own GGUF files. On October 31 it added INT4, INT8 and FP8 checkpoints, but in the compressed-tensors safetensors format, which llama.cpp and Ollama do not load; the INT4 one was built with quantization-aware training, according to its model card, and the community GGUF files were not. Hugging Face lists the Q4_K_M file as 20.79 GB because it counts in decimal units; the tables here use binary gigabytes, like our calculator, which gives 19.36 GB for the same file.

How to run it

ollama run hf.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q4_K_M
llama-cli -hf bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q4_K_M -ngl 99 -c 4096 --temp 0.6 --top-p 0.95

There was no Nanbeige entry in the Ollama library on November 2, so the hf.co form is the way to pull it in Ollama. The quantizer built the files with llama.cpp release b11845. Set the context explicitly (-c in llama.cpp, num_ctx in Ollama): it is the setting that decides whether the model stays on your GPU. The temperature and top_p values above are the ones the model card suggests for reasoning and chat; for agent and tool-use tasks it suggests a temperature of 1.0.

Thinking is on by default. The model card documents an enable_thinking switch in the chat template to turn it off.

Which one should you pick?

On 24 GB, take Q4_K_M for chat at short contexts, and IQ3_M if you need 32K for coding or agent work. On 32 GB, Q5_K_M covers both cases. On 16 GB, Q3_K_S or IQ3_XXS run at 4K only, a quality level the site rates "Fair"; compare with a smaller model such as Qwen3.8 before you settle. The model card presents the 33B as the flagship "teacher" of the 4.3 series, aimed at coding and office-agent tasks, and reports 80.9 on Terminal-Bench 2.1 and 60.3 on SWE-Bench Pro, where its own table shows Qwen3.8-27B slightly ahead. Those are the vendor's numbers, not ours.

Limits

• License: Apache 2.0, read in the Hugging Face tag and the model card header; the repository has no separate license file. It is not gated.
• The GGUF files come from a community quantizer, not from the Nanbeige team.
• Speed is an estimate only: the calculator gives an estimated 20.7 tokens per second for Q4_K_M fully on the GPU, and an estimated 2.6 with offload. Our measurement on an 8 GB laptop GPU found real speeds between 23% and 81% of the estimate, and the editor has not run this model.
• The model card names a Nanbeige4.3-3B as a future target of its distillation pipeline. No such repository existed on November 2, and this guide does not cover it.

See every quantization on the Nanbeige4.3-33B model page, check it against your own GPU, or browse the best LLMs for 24 GB of VRAM. Source: the official model card and its config.json.

How we made this: file sizes and architecture were read from the public Hugging Face API on November 2, 2026; memory totals were computed with the AI Local Check calculator (see our methodology); the draft was written with AI assistance and reviewed by the editor before publication.