Running Kimi K3 locally: status and realistic math
Last updated:
Status: weights not yet released. Moonshot announced Kimi K3 on July 16, 2026 and promised open weights by July 27, 2026. When they land, K3 — at ≈2.8T parameters — will be the largest open-weight model ever published, and "local" will mean something different than it did for 7B models. The capacity math below is unforgiving, but it beats finding out from a stalled download.
Release timeline
| Date | Event |
|---|---|
| 2026-07-16 | K3 announced; API + kimi.com access live |
| ≤ 2026-07-27 | Open-weight release promised (Hugging Face expected) |
| after release | Community quantizations (GGUF/MLX), vLLM/SGLang support, GPU-cloud hosting — tracked here as they ship |
Estimated storage & memory (pre-release math, clearly labeled)
These are arithmetic estimates from the published 2.8T parameter count, not vendor specs. Weight storage scales linearly with precision:
| Precision | Approx. weights on disk | Practical meaning |
|---|---|---|
| BF16 | ≈5.6 TB | research clusters only |
| FP8 | ≈2.8 TB | multi-node GPU serving (likely the official serving format) |
| INT4 / Q4 GGUF | ≈1.4–1.6 TB | terabyte-class download; high-RAM server + offload = slow but possible |
Because K3 is a mixture-of-experts, active parameters per token are far fewer than 2.8T (Moonshot hasn't published the number; K2 was 1T total / 32B active, and K3 likely follows similar sparsity). MoE sparsity cuts compute per token, not memory: you still need the full expert set resident or streamed. That is why realistic "local" K3 for individuals probably means renting a multi-GPU node by the hour rather than home hardware.
What to prepare before July 27
- Follow moonshotai on Hugging Face so the drop shows in your feed immediately.
- Decide your serving story now: vLLM or SGLang on rented H200/B200-class nodes are the likely first-supported paths, based on how K2's release played out.
- Budget the transfer: even a Q4 file set is a 1.4TB+ download — plan storage and bandwidth ahead.
- Have an eval set ready: your own 20-prompt suite beats launch-week benchmark debates — see our benchmark caveats.
Meanwhile: the API is the local dev loop
Until weights land, the pragmatic setup is the API with aggressive prompt caching — functionally similar iteration speed, zero infrastructure. The API guide covers setup in ten minutes, and the cost calculator will tell you whether self-hosting could ever beat $3/$15 for your volume once GPU-rental prices for K3 emerge. For most workloads under a few billion tokens a month, it won't — but the calculator does that math without wishful thinking.
Frequently asked questions
Can I run Kimi K3 locally right now?
Not yet. As of July 21, 2026 the weights have not been published; Moonshot promised an open-weight release by July 27, 2026. Until then K3 is API/app-only. This page updates the day the weights land.
What hardware will Kimi K3 need?
Nobody outside Moonshot knows exactly, but a 2.8T-parameter MoE implies roughly 2.8TB of weights at FP8 and ~1.4TB at 4-bit — multi-node GPU territory, or very-high-RAM servers for slow CPU/offload inference. Single consumer GPUs are out of the question for the full model; expect distilled or pruned community variants to be the realistic local path.
Will there be a GGUF of Kimi K3?
Almost certainly — the community produced GGUF quantizations for Kimi K2 within days of its release. File sizes will be enormous (a terabyte-class download even at aggressive quantization), so hosted inference on rented GPUs will be the practical route for most people.