Kimi K3 fits 2.8T parameters into production hardware. Here is the architecture stack.

Open weights are easy to celebrate until you stare at the download size. Kimi K3 from Moonshot AI ships 2.8 trillion parameters in an open checkpoint. That is frontier-class scale with a five-terabyte BF16 problem if you serve it naively.

The benchmark headlines are real. K3 ranks third on the Artificial Analysis index and punches hard on agentic knowledge work. What I keep coming back to, after reading the technical report and the AlphaSignal deep dive, is simpler: Moonshot designed this model to run, not just to win slides.

If you are routing long-context agents or evaluating whether open weights can replace closed APIs, the architecture choices matter more than another leaderboard row.

MoonshotAI/Kimi-K3 on GitHub

By the numbers

K3 is a native multimodal MoE transformer. Moonshot's model card lists the core specs:

Moonshot claims roughly 2.5x better scaling efficiency versus Kimi K2. That is not a vibes metric. It is the payoff from co-designing routing, attention, and training precision so each FLOP buys more capability.

On Artificial Analysis, K3 sits behind only the closed frontier leaders on the aggregate index. On AA-Briefcase, an agentic knowledge-work suite, it lands second overall, ahead of GPT-5.6 Sol (max) and Claude Opus 4.8 on several rows Moonshot publishes.

Those scores are vendor-run. Treat them as directional. The deployment story is harder to fake.

Stable LatentMoE: routing without melting the network

Standard MoE saves compute by activating a slice of experts per token. At 2.8T scale, the bottleneck shifts. Moving full-width activations between experts burns memory bandwidth faster than matmuls do.

K3's Stable LatentMoE compresses tokens into a 3,584-dimensional latent space before routing. Experts still hold 3,072-dim hidden states, but the cross-node traffic shrinks because routing decisions happen in the smaller latent manifold.

This builds on ideas Nvidia explored in Nemotron 3 and the broader LatentMoE research line. Open labs borrowing from each other's routing tricks is how trillion-parameter serving stops being science fiction.

For applied teams, the lesson is blunt: your MoE bill is mostly networking. If you are self-hosting, profile all-to-all before you obsess over FLOPs.

Hybrid attention: breaking the KV cache wall

A 1M-token context with standard multi-head attention is a memory horror show. Every layer caches keys and values that grow linearly with sequence length. Multiply by 93 layers and you are planning data-center budgets, not a product launch.

K3 answers with two mechanisms Moonshot stitches together:

1. Kimi Delta Attention (KDA): a linear/recurrent attention variant that keeps a fixed-size state updated per token instead of an ever-growing KV tensor.
2. Gated MLA: DeepSeek-style latent KV compression that stores a single compressed vector per token and expands during inference.

The stack runs about three KDA layers for every one Gated MLA layer. KDA handles cheap long-range mixing. MLA restores high-fidelity retrieval when the model needs to pinpoint an exact span in a million-token pile.

Moonshot open-sourced FlashKDA kernels for this path. That is the kind of detail that separates a paper model from something vLLM can actually schedule on day zero.

If you run client RAG pipelines with 200K+ token contexts, this hybrid pattern is the blueprint: pay for full attention periodically, not everywhere.

Quantization-aware training at FP4

Raw BF16 weights for 2.8T parameters exceed 5 TB. LatentMoE and KDA shrink runtime memory, but the checkpoint still has to land somewhere.

K3 trains with quantization-aware training (QAT) in mixed precision: MXFP4 weights and MXFP8 activations. The model learns under low-precision numerics during post-training, so accuracy loss stays bounded when you deploy in 4-bit.

Moonshot reports the on-disk footprint near 1.4 TB at native MXFP4, a much smaller ratio than parameter count alone would suggest.

Unsloth's follow-up experiments pushed public GGUF builds down toward 594 GB at 1–2 bit dynamic quantization while keeping roughly 79% top-1 accuracy on their spot checks. That is not production-ready for everyone, but it shows the QAT foundation is not cosmetic.

When a client asks "can we run this on-prem cheaply," I start with which quantization recipe they mean, not the parameter count on the slide.

Accuracy patches: AttnRes and NoPE

Extreme sparsity plus 4-bit weights usually costs reasoning quality. Moonshot added two stabilizers worth naming.

Attention Residuals (AttnRes) replace the classic "add layer output to input" residual with a learned attention over prior layer representations. Layers can pull signal from depth without diluting it through 90 serial additions.

NoPE (no rotary positional embeddings) drops RoPE entirely. K3 instead accumulates implicit position signal through its other architectural choices, which Moonshot ties to cleaner million-token behavior without the long-context rot some RoPE stacks show.

I am still skeptical of any "we removed position embeddings and nothing broke" claim until independent long-context evals reproduce it. The engineering intent is clear: remove components that fight extreme length.

What the ecosystem already ships

Frontier open weights are useless if inference frameworks lag. K3 launched with day-zero containers from partners targeting multi-node tensor parallelism over NVLink on GB300-class clusters.

Practical entry points today:

• Kimi API quickstart (kimi-k3 model id, OpenAI-compatible SDK)
• GitHub weights + technical PDF
• arXiv technical report
• vLLM / SGLang integrations (check your provider's latest image tags)

API pricing from Moonshot's launch materials sits around $0.30 / $3.00 / $15.00 per million tokens across input, output, and thinking tiers, with the 1M context window available on the platform path. Self-host math depends on your quantization, expert parallelism, and how often agents actually fill the context window.

How this differs from my Kimi K3 sandbox post

I already wrote about K3 cheating a UK cyber benchmark by cloning a public GitHub repo inside a misconfigured sandbox. That story is about eval hygiene and open-weight distribution.

This post is the other half: why the model is deployable at all. Same checkpoint, different lens. Capability and containment both belong in your vendor review.

What I would test before routing production traffic

Moonshot did the hard systems work. Your job is still a harness review:

1. Context fill rate: agents rarely need 1M tokens. Measure p95 prompt length before you pay for max context.
2. Expert parallelism config: 896 experts reward clusters with fast interconnects. Single-node fantasies die here.
3. Quantization tier: API MXFP4 vs community GGUF is not the same product.
4. Task-specific eval: run your own coding or RAG suite; AA-Briefcase is a hint, not your SLA.

K3 is the clearest proof yet that open frontier models are architecture competitions. Parameter count is the headline. Latent routing, hybrid attention, and native QAT are why the headline ships.

If you are comparing open MoE stacks for agent workloads and want a practical routing review, book a free discovery call. I focus on inference cost and harness fit before model religion.