Can You Run Ling-3.0-flash Locally?
By Lefi Abdelmonem · LinkedIn ↗
Author · AI Local Check · Published August 8, 2026
Here's a sentence that wasn't possible a year ago: you can run a 127-billion-parameter model on a single consumer graphics card. That's the story of Ling-3.0-flash, released August 2, 2026 by inclusionAI (Ant Group) under the MIT licence — and it's possible because of how the model is built, not magic. Every number here is measured from the real GGUF files.
Why a 127B model can fit a 32 GB GPU
Ling-3.0-flash is an extreme Mixture-of-Experts model: 512 experts, of which only 8 run per token. That sparsity means its weights compress unusually well, and at a 1-bit quantization the whole model comes down to about 31 GB — small enough to sit entirely in the VRAM of a single 32 GB RTX 5090.
Two design choices make this work:
- Ultra-sparse MoE (8 of 512 experts). Only a tiny fraction of the network runs on each token, so it's fast — though remember, all 512 experts still have to be held in memory. The 31 GB already reflects the full model, not the active slice.
- Multi-head Latent Attention + hybrid layers. Ling caches a single compressed latent per layer instead of full keys and values, and mixes in linear-attention layers. The result: a tiny KV cache — about 0.18 GB at 4K tokens, rising to only ~6 GB at its full 131K context. Long context is nearly free.
How much VRAM does Ling-3.0-flash need?
Measured from the real GGUF files, at a 4,096-token context:
| Quantization | Memory to run | Runs on |
|---|---|---|
| IQ1_S (~1-bit) | ≈ 31 GB | a single 32 GB GPU (RTX 5090) |
| IQ1_M | ≈ 35 GB | 32 GB + a little offload |
| IQ2_M (~2-bit) | ≈ 47 GB | 48 GB GPU, or a 64 GB machine |
| IQ3_XXS | ≈ 54 GB | 64 GB machine |
| Q4_K_M (~4-bit) | ≈ 75 GB | 128 GB unified-memory machine |
| Q6_K | ≈ 101 GB | 128 GB machine |
| Q8_0 | ≈ 125 GB | 128 GB machine |
Figures include weights, KV cache and a system margin.
So, can you run it?
- 32 GB GPU (RTX 5090): yes, at 1-bit (IQ1_S ≈ 31 GB), entirely in VRAM — a genuine first for a 127B model on one card.
- 24 GB GPU (RTX 3090/4090): not fully in VRAM, but it runs by offloading the overflow to system RAM (slower). With 64 GB of RAM you're fine.
- 128 GB unified-memory machine (Apple Silicon, AMD Strix Halo): the sweet spot — run it at Q4 (≈ 75 GB) for far better quality than the 1-bit build.
The specs
• Licence: MIT
• 127.5B total parameters, MoE (512 experts, 8 active per token)
• Architecture: hybrid attention with Multi-head Latent Attention (small KV cache)
• Native context: 131,072 tokens
Check it against your own hardware
See the live figures for every quantization on the Ling-3.0-flash model page, or enter your own GPU to see exactly which build fits. New to the labels? Start with GGUF quantization explained, and see how Ling compares to other big models in DeepSeek V4 and Hy3.
The bottom line
Ling-3.0-flash is a milestone for local AI: a 127B, MIT-licensed model that, thanks to extreme MoE sparsity and latent attention, fits a single 32 GB GPU at 1-bit and runs at high quality on a 128 GB machine. If you have an RTX 5090 or a large unified-memory box, this is one of the biggest models you can genuinely run at home. Check what your PC can run.