How much VRAM does Ornith-1.5-35B-A3B need?

Measured from the real GGUF files, about 21.1 GB at Q4_K_M and 28.1 GB at Q6_K, including KV cache and system margin at a 4,096-token context. The published builds start at Q4, so roughly 21 GB is the floor, which makes a 24 GB card the entry point. A 16 GB card runs it only by offloading to system RAM.

Does Ornith-1.5-9B run on a small GPU?

Yes. At 9.7 billion parameters it runs on an 8 GB GPU at about 8 GB (Q6_K), keeping most of the quality, and a 12 GB card can run the full Q8_0 at about 10 GB. Its hybrid attention also keeps long-context memory low, so it is a solid choice for a laptop or older desktop GPU.

Is Ornith-1.5-35B-A3B fast, and does long context cost much memory?

It is fast: as a Mixture-of-Experts model it activates only about 3 billion of its 36 billion parameters per token. And its hybrid attention keeps context cheap: only 10 of its 40 layers hold a full KV cache, so the cache is about 0.08 GB at 4K tokens and only 5 GB at its full 262K context. All 256 experts still occupy memory, so the ~21 GB footprint reflects the whole model.

Can You Run Ornith 1.5 Locally?

Q: Can you run Ornith 1.5 locally?

Yes, depending on the size. Ornith-1.5-9B runs on an 8 GB GPU (about 8 GB at Q6_K). Ornith-1.5-35B-A3B, a Mixture-of-Experts model, fits a 24 GB card at Q4_K_M (about 21 GB) and a 32 GB card at Q6_K. The 397B flagship needs a multi-GPU server or large unified-memory machine. All three are MIT-licensed. Figures are measured from the real GGUF files.

By Lefi Abdelmonem · LinkedIn ↗

Author · AI Local Check · Published August 25, 2026

Here's something odd about Ornith 1.5. If you follow AI closely, you've probably read a dozen headlines this month about Kimi, DeepSeek and Qwen — and maybe none about ornith-ai. Yet when I checked the download counters this week, the Ornith 1.5 GGUF builds had quietly crossed a million downloads each. Two of them. A model family barely anyone is writing about is one that an awful lot of people are actually running.

So I pulled the real files and measured the whole lineup on a range of hardware. The short version: if you've got a mainstream GPU, there's an Ornith that fits you — and it's MIT-licensed, which makes the story even better.

Three sizes, and which one is yours

Ornith 1.5 comes in three flavours, all released August 18 and all built on the Qwen3.5 architecture:

Ornith-1.5-9B — the little one that runs almost anywhere.
Ornith-1.5-35B-A3B — a Mixture-of-Experts model, and the one most people should care about.
Ornith-1.5-397B — the flagship, and the one you'll admire from a distance unless you own a server.

Let me take them in order, because the answer to "can I run it?" changes a lot between them.

Ornith-1.5-9B: the easy yes

This is the one to grab if your GPU is ordinary. At 9.7 billion parameters it's small enough that even an 8 GB card runs it comfortably — about 8 GB at Q6_K, which keeps almost all of the quality. If you've got 12 GB, you can run the full-fat Q8_0 (~10 GB) and not think about it again.

It also inherits the family's hybrid attention, so long chats stay cheap in memory. For a laptop GPU or an older desktop card, this is a genuinely capable model that just works.

Ornith-1.5-35B-A3B: the sweet spot

This is the interesting one. It's a Mixture-of-Experts model — 256 experts, but only 8 fire on any given token — so despite its 36 billion parameters it runs at the speed of something much smaller. The "A3B" is the tell: roughly 3 billion active parameters per token.

Measured from the real GGUF files, here's where it lands:

Your GPU	Best build that fits	Memory
16 GB	— (offloads to RAM)	21+ GB
24 GB (3090/4090/5090)	Q4_K_M	≈ 21.1 GB
32 GB (RTX 5090)	Q6_K	≈ 28.1 GB

One honest caveat: the published GGUF builds start at Q4 — there are no aggressive 2-bit versions — so 21 GB is effectively the floor. That puts a 24 GB card as the real entry point; a 16 GB card runs it only by spilling into system RAM, which works but drags.

The payoff for the 24 GB crowd is that context is nearly free. Only 10 of its 40 layers keep a full attention cache; the rest use linear attention. In numbers, the KV cache is about 0.08 GB at 4K tokens and only 5 GB at the full 262K — so you can feed it long documents without watching your VRAM evaporate. That's the same trick that makes KAT-Coder so pleasant to run, and it's why a 30B-class MoE feels lighter than its size suggests.

Ornith-1.5-397B: look, don't touch (yet)

The flagship is 403 billion parameters. Even at 4-bit it wants roughly 229 GB, and a 48 GB workstation card still can't hold it without heavy offloading. This is multi-GPU-server or large-unified-memory territory, in the same weight class as DeepSeek V4. If that's your hardware, wonderful; for everyone else, the 35B is the one that punches closest to it while actually fitting on a card.

A quick reminder on MoE, because it trips people up: "3B active" makes Ornith-35B fast, not small. All 256 experts still have to sit in memory — the 21 GB figure already counts every one of them. Active-parameter counts buy speed, never a smaller footprint.

So, is it worth it?

If you value a permissive licence, Ornith 1.5 is unusually generous — it's MIT, which is about as free as open weights get, and rarer at this size than you'd think. Pair that with the adoption numbers and it's clearly resonating with people who run models locally rather than just benchmark them in the cloud.

My take, having measured all three: grab the 9B if you're on 8–12 GB, run the 35B-A3B if you have a 24 GB card and want something that feels a tier above its speed, and leave the 397B to the server owners. Whichever you pick, the honest way to be sure is to check it against your own GPU — the numbers here come straight from the files, and yours might differ once you set your real context length.

See every quantization live on the Ornith-1.5-35B-A3B model page or the 9B page, and if you're weighing it against the rest of this year's crop, the best local LLM guide lays them out side by side.

→ Check what your exact GPU can run