Can You Run Inkling Locally?

Updated August 1, 2026 — there's now a version that fits. Thinking Machines has released Inkling-Small, a 266-billion-parameter sibling under Apache 2.0. We measured it: it runs on a 128 GB unified-memory machine, and not only at 1-bit. Jump to the Inkling-Small numbers.

Inkling is the first model from Thinking Machines, the lab founded by former OpenAI CTO Mira Murati — and it landed on July 15, 2026 as an open-weight release. It's already on Hugging Face as GGUF, which raises the obvious question: can you run Inkling locally?

The honest answer for the original model: not on a normal PC — it is enormous. But the smaller sibling released since then changes that answer, and it's the one most people should look at.

What is Inkling?

Inkling is a Mixture-of-Experts (MoE) model with close to 950 billion parameters (around 41 billion active per token), trained to reason natively across text, image, audio and video. It is open-weight, so anyone can download it — the real constraint is whether your hardware can hold it in memory.

How much memory does Inkling need?

Computed from its real GGUF files, even the most aggressive quantizations are far beyond a desktop:

Even at an extreme 1-bit quantization, Inkling needs about 226 GB — more than a stack of RTX 5090s. It's designed for multi-GPU servers or large unified-memory machines, not a gaming PC.

Inkling-Small: the version that actually fits

On July 27, 2026, Thinking Machines published Inkling-Small — same family, 266 billion parameters instead of 950, under a plain Apache 2.0 licence. That is the release most people should care about, because it crosses back into reach of high-memory workstations.

Measured from the real GGUF files (unsloth's build), at a 4,096-token context:

The important detail: you're not limited to a desperate 1-bit squeeze. A 128 GB unified-memory machine — AMD Strix Halo, a 128 GB Mac, NVIDIA's RTX Spark — runs Inkling-Small all the way up to Q3_K_M at ~113 GB, which is a genuinely usable quality level. Compare that with the original Inkling, where even 1-bit demanded 226 GB and nothing fit.

It sits in the same bracket as Tencent's Hy3 (~87 GB at 1-bit): too big for any single graphics card, but within reach of the new class of large unified-memory boxes. Adoption reflects that — the NVFP4 and GGUF conversions have together pulled well over 40,000 downloads within days of release.

What can you run instead?

If you want a capable model on an ordinary graphics card — not a 128 GB workstation — plenty of options fit. See the best LLM for your VRAM for size-by-size picks, or KAT-Coder if you want a strong coding agent on a 24 GB card. The original Inkling stays in the "too big for home" tier alongside Kimi K3 and DeepSeek V4.

The bottom line

The original Inkling (~950B) remains server-class: ~226 GB even at 1-bit. But Inkling-Small changes the picture — 266B under Apache 2.0, running at up to Q3_K_M in about 113 GB, which a 128 GB unified-memory machine handles. If you own that class of hardware, it's one of the strongest open models you can actually host. If you're on a normal GPU, check what your PC can run and pick something that fits.