Mixture-of-Kittens: Cursor open-sourced the MoE megakernel that trains Composer

The MoE layer can eat more than half of end-to-end training time on agent-scale models. That is not a rounding error. It is the bill you pay every step while the rest of the stack waits on expert routing and all-to-all comms.

Cursor open-sourced Mixture-of-Kittens (MoK) on August 5, 2026 under Apache 2.0. It is the production MoE training megakernel that now powers Composer across tens of thousands of GPUs. The repo lives at github.com/cursor/mixture-of-kittens.

I do not train frontier models in my garage. I do help teams reason about where agent economics come from: routing, inference kernels, and the boring infra that decides whether a coding model ships on schedule or burns rack hours. MoK is the infra layer most practitioners never see, and it is worth understanding even if you will never touch an NVL72.

What MoK is in plain language

Mixture-of-Kittens fuses all MoE communication and computation into a single, fully deterministic megakernel built for NVL72 racks. Instead of dispatch all-to-all, expert FFN, combine all-to-all, and CPU-side buffer sizing running as separate phases with sync points, MoK overlaps compute and networking inside one kernel launch.

That matters because expert-parallel MoE is half math and half traffic control. Tokens scatter to the GPUs holding their assigned experts, run grouped GEMMs, then gather back for a weighted sum. When comms and compute run sequentially, you pay for both at full wall-clock cost.

MoK targets DeepSeek-V3-style MoE layers used across GLM, Qwen, Kimi, and the Kimi 2.5 base that feeds Composer 2.5. Shared expert plus hundreds of routed experts is the pattern.

Why NVL72 changed the problem

Cursor moved training to GB300 NVL72s. Two constraints showed up fast.

First, an NVL72 is a multi-node rack inside one NVLink domain. All 72 GPUs can overlap fine-grained compute and communication without treating the rack like a loose cluster.

Second, the integrated Grace CPUs are slow relative to Blackwell GPUs. GPU streams catch up to CPU-side scheduling work, then sit idle. Cursor's traces showed kernels blocked on CPU logging and metrics pushes, problems that barely appeared on DGX boxes with faster host CPUs.

MoK responds by minimizing CPU work and eliminating CPU-GPU synchronization. Ring token buffers replace the usual choice between token dropping (bad for quality) and exact buffer allocation on the host (bad for throughput).

Practitioner read: If your training stack assumes fast host CPUs for MoE scheduling, Blackwell NVL72 is a different machine. Kernels that looked fine on Hopper plus Xeon can stall hard here.

Push vs pull, and why the megakernel shape matters

Cursor's blog walks through communication direction in detail. The short version: push-based dispatch is the default in many stacks because it supposedly saturates links better. MoK uses pull-based dispatch and push-based combine, reusing one schedule table for all four comms ops across forward and backward.

Pull-based dispatch drops cross-GPU signalling overhead. In Cursor's multi-node microbenchmarks, push-based dispatch signalling ran about 5.8x higher latency than pull (103 µs vs 18 µs). That cost compounds inside a megakernel where launch boundaries already hurt.

The megakernel itself schedules SM tasks with CLC (Cluster Launch Control), a Blackwell feature for persistent grids that can yield to higher-priority streams. That lets intra-rack MoK overlap with inter-rack FSDP all-gather without serializing behind one giant kernel.

For MXFP8, MoK fuses activation quantization into dispatch, grouped GEMMs, and SwiGLU. Weights can be pre-quantized with a dedicated kernel. Cursor trains in MXFP8 for speed and reports no numerical issues, keeping the shared expert in BF16 for stability.

Benchmark numbers worth pinning

Cursor published layer-level benchmarks (code in the repo benchmarks/ folder) and internal end-to-end numbers on 512 GPUs across GB300 NVL72 racks. EP degree 64, 2,048 tokens per GPU before routing.

Baselines included NCCL + PyTorch, DeepEP + PyTorch, DeepEP + TransformerEngine, and HybridEP + Megatron (NVIDIA's recommended NVL72 option over DeepEP + Megatron).

End-to-end on Cursor's production stack (DeepEP MoE layer swapped for MoK):

Those are not synthetic microbench wins. They are the kind of delta that changes how many Composer-scale training runs fit in a quarter.

What this means if you are not on an NVL72 rack

Be honest about the hardware gate. MoK requires Blackwell SM100 or SM103 in NVL72 form factors. If you train MoE on Hopper or a cloud SKU without NVLink domains at rack scale, you cannot drop this in tomorrow.

You still get value from the release:

• Reference design for overlapping MoE comms and compute without CPU sync
• Reproducible benchmarks against public DeepEP and HybridEP stacks
• Signal on Composer's training path, which connects to the model you already route in Cursor daily

If you read my post on Cursor Router's Intelligence, Balance, and Cost modes, Router optimizes inference spend per request. MoK optimizes training throughput per dollar on the supply side. Same company, opposite ends of the agent model lifecycle.

For open MoE inference economics, compare with Nemotron 3.5 Lightning: 30B total, 3B active, built for fast agent execution loops. MoK is not an inference stack. It is the training kernel that makes shipping those MoE architectures at Cursor scale feasible.

Install sketch (when you have the iron)

From the README, a minimal verify path after matching PyTorch to CUDA 13.0:

pip install "torch==2.10.0+cu130" --index-url https://download.pytorch.org/whl/cu130
pip install . --no-build-isolation
python -c "import mok; print(mok.__version__)"

Multi-GPU tests run through torchrun and pytest. Default builds target SM103; set MOK_ARCH=SM100 for GB200.

Cursor also notes they wrote much of MoK with agent assistance on single-operator kernels and the distributed megakernel itself, without leaning on a simplified framework layer. That matches how small teams now ship systems code: humans on architecture, agents on boilerplate and iteration speed.

What I would watch next

Three follow-ups I am tracking:

1. Community ports to other MoE configs beyond DSV3-style layers
2. HybridEP parity as NVIDIA and Cursor both push NVL72-native paths
3. Inference bridge: Cursor already ships warp decode for MoE inference; training and serving kernels aligning would shrink the full Composer pipeline story

MoK will not change your Tuesday standup if you only consume models through APIs. It will matter if you train MoE agents, benchmark rack utilization, or want to see how Cursor keeps Composer competitive without hiding the infra.

Planning agent model economics across training, routing, and inference? Book a free discovery call. I help teams map where spend actually lives, from megakernels to request-level routers.