Most agent stacks waste compute by running a huge dense model on every boring step: tool selection, JSON formatting, log parsing, retry logic.
Nemotron 3.5 Lightning is NVIDIA's answer for the execution layer: a 30B mixture-of-experts (MoE) model that only activates 3B parameters per token.
I route models in client agent loops every week. This release is aimed exactly at the tier where cost per task matters more than leaderboard vanity.
MoE in one sentence for practitioners
A router sends each token to a small subset of expert layers. Total capacity stays large. Per-token compute stays small.
Think of a company where only the right specialists join each meeting instead of the entire org chart.
| Spec | Nemotron 3.5 Lightning |
|---|---|
| Total parameters | 30B |
| Active per token | 3B |
| Architecture | Mamba-2 + MoE + selective attention |
| Context | Up to 1M tokens |
| Release | August 11, 2026 |
| License | OpenMDW 1.1 |
Built for harnesses, not just chat
NVIDIA positions Lightning for always-on agents running through harnesses like OpenClaw and Hermes Agent, with management via the NeMoClaw open stack.
Training targets high-volume agent behaviors: tool calling, reasoning steps, and sub-agent delegation. That is different from a model tuned only for single-shot chat completions.
Benchmark claims from the release post:
- ~4x faster output than similar-sized models
- 86% on PinchBench
- 10,000 tasks completed 35% faster than Qwen3 35B in their harness comparison
- Runs on a single H100 or DGX Spark
Pin your own evals before you trust vendor numbers. The architecture story still holds: MoE plus speculative decoding plus harness-aware training is the 2026 pattern for agent economics.
Speculative decoding and NVFP4
Lightning ships with multi-token prediction (MTP) from training plus inference options like DFlash and DSpark for speculative decoding.
There is an NVFP4 checkpoint alongside BF16 using the same specialized kernels as Nemotron 3 Ultra across Blackwell, Hopper, and Ampere GPUs. One quantized file can serve data center and desktop Spark deployments.
Recommended sampling from the NIM model card: temperature 1.0, top_p 0.95.
How to run it locally
NVIDIA documents a NIM container with OpenAI-compatible and Anthropic-compatible APIs:
docker pull nvcr.io/nim/nvidia/nemotron-3.5-lightning-30b-a3b:2.0.9-variant
For reasoning output, enable the Nemotron v3 parser:
docker run --gpus all -p 8000:8000 \ nvcr.io/nim/nvidia/nemotron-3.5-lightning-30b-a3b:2.0.9-variant \ --reasoning-parser nemotron_v3
Point LangChain, LangGraph, LlamaIndex, or your custom harness at http://localhost:8000/v1.
Weights and cards:
- build.nvidia.com model card
- NIM get-started guide
- Hugging Face release (August 11, 2026)
Fine-tune with NVIDIA NeMo for domains like cybersecurity, coding, or legal workflows.
Where I would slot it in a routing stack
| Loop phase | Model tier |
|---|---|
| Tool calls, file edits, log triage | Nemotron 3.5 Lightning or similar MoE workhorse |
| Architecture decisions, novel bugs | Frontier model on reserve |
| Long session memory | Lightning's 1M context or external memory layer |
This mirrors how I already advise teams on Composer-class workhorses plus Opus on demand. Lightning is another open option in that middle tier, especially when you own GPUs or Spark boxes.
Pair with posts on agentic coding model routing and specialized model economics for the full cost picture.
Takeaway: Nemotron 3.5 Lightning is not trying to be the smartest model in the room. It is trying to be the fastest honest worker in a 24/7 agent loop. That is the layer where margin lives.
Tuning agent loops for latency and token spend? Book a free discovery call. I help teams benchmark harnesses, pick workhorse models, and keep frontier tiers in reserve.