Everyone quotes DeepSeek's training budget. Almost nobody reads the code that makes serving those models affordable at scale.
The AlphaSignal signal pointed at DeepSeek's infrastructure papers as "the biggest open-source AI contribution this century." That is hype. The open-infra-index drop is still the most important infra release I have seen from a model lab in years, because it attacks the layer most teams outsource and overpay for.
open-infra-index documents production-tested components DeepSeek uses online, released as a one-repo-per-day series starting February 24, 2025.
The five repos that actually move the needle
| Project | What it does |
|---|---|
| FlashMLA | MLA decode kernels for Hopper GPUs, paged KV cache |
| DeepEP | Expert-parallel all-to-all comms for MoE training and inference |
| DeepGEMM | Lightweight JIT FP8 GEMM for dense and MoE matmul |
| DualPipe + EPLB | Pipeline parallelism overlap and expert load balancing |
| 3FS | High-performance distributed file system for training IO |
These are not tutorial kernels. DeepSeek positions them as battle-tested in production, with ISCA/SC papers backing the hardware co-design story.
Why MoE infra is the hidden product
Mixture-of-Experts models win on quality per flop by activating a subset of experts per token. They lose if your all-to-all communication is slow or your expert load is skewed.
DeepEP is the piece I watch closest. It implements high-throughput and low-latency GPU kernels for MoE dispatch/combine, with FP8 support and hooks for overlapping communication with compute. That is the unglamorous work separating "MoE on a slide" from MoE at 2am when traffic spikes.
DeepGEMM complements it with ~300 lines of JIT-compiled FP8 GEMM focused on the shapes DeepSeek actually runs, not generic BLAS nostalgia.
For applied AI engineers who will never touch CUDA, the lesson is still practical: your model bill is infra-shaped. Two teams running the same checkpoint can pay wildly different inference costs depending on kernels, batching, and EP layout.
Papers worth bookmarking
Two papers anchor the release beyond GitHub stars:
- Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures (ISCA 2025 industry track)
- Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning (SC24)
Read these if you are arguing about build vs rent GPU capacity with a CFO who only sees model API line items.
What this does not solve for most SaaS teams
Unless you self-host large MoE models, you will not install DeepEP next sprint. Cloud APIs hide this layer behind a per-token price.
Where it still matters:
- Vendor selection: ask providers what EP/GEMM stack they run; vague answers are a yellow flag
- Fine-tuning and distillation shops: kernel efficiency changes how many experiments you can afford
- Future on-prem agents: if open-weight MoE keeps growing, infra literacy returns
Useful entry point: DeepSeek open-infra-index
The takeaway
Models get the keynote. Infra gets the margin. DeepSeek giving away production kernels is a bet that open ecosystems move faster than sealed datacenter magic.
If you are planning GPU spend, agent hosting, or private model deployment and want help separating benchmark hype from serving economics, book a free discovery call.