Nvidia has spent two years arguing that the future of AI is a system of models rather than one oracle you route everything through. On August 11, it shipped the piece that argument was missing.
Nemotron 3.5 Lightning is an open 30B mixture-of-experts model with 3B active parameters, and Nvidia is refreshingly blunt about what it's for. Not benchmarks. Not chat. It's built for what the company calls the execution layer of always-on agents: tool calls, result validation, subagent delegation, formatting, the git pull of agentic work.
It is the smallest member of the Nemotron 3 family, and it may be the most useful one.
The problem it actually solves
If you have run a long-horizon coding agent, you already know the shape of the bill. The model spends a few thousand tokens planning and then a few hundred thousand tokens doing bookkeeping — reading a file, checking whether a command succeeded, reformatting a result, handing a subtask to a child agent.
Sending every one of those steps to a frontier reasoning model is, as Nvidia's Chris Alexiuk and Chintan Patel put it, a way to add "cost and latency" to work that requires neither. The proposed division of labor is explicit:
Plans route up to the frontier, execution routes down to Lightning.
That routing is not left as an exercise for the reader. Alongside the model, Nvidia released NeMo Switchyard, a library that exposes Lightning as a routing target next to your open and closed models so each request lands on the cheapest model that can actually handle it. Frontier reasoning models like Nemotron 3 Ultra keep orchestration and complex planning; Lightning takes the volume.
The numbers
Nvidia's headline claim is a throughput claim, and it's the right one to make for this class of model.
| Metric | Nemotron 3.5 Lightning |
|---|---|
| Total / active parameters | 30B / 3B |
| PinchBench accuracy | 86% |
| 10,000-task completion time | 30% faster than Qwen3.6 35B at similar accuracy |
| Output speed | Up to 4x similar-sized models |
| Checkpoints | NVFP4 and BF16 |
| License | OpenMDW-1.1 |
The PinchBench figure is the one worth dwelling on. Nvidia frames agent efficiency as "how quickly a model completes useful work and not simply how fast it generates tokens" — a distinction that quietly indicts most model marketing. Lightning hits 86% accuracy on PinchBench while finishing 10,000 tasks 30% faster than Qwen3.6 35B at comparable accuracy. That's the same answer, sooner.
On the Artificial Analysis Intelligence Index — an aggregate of nine evaluations spanning agentic tasks, coding, scientific reasoning, and general intelligence — Nvidia claims Lightning wins the accuracy-versus-speed Pareto frontier for small open models.
Where the speed comes from
Two mechanisms, both familiar to anyone who has read Nvidia's inference work.
Speculative decoding. Multi-token prediction was baked in during a dedicated pretraining stage, as with Nemotron 3 Super and Ultra, then sharpened in a separate MTP-boosting phase. Nvidia also ships two draft models: DSpark, recommended for DGX Spark and low-concurrency data center workloads, and DFlash, which may win on your specific serving profile. MTP itself is best at medium-to-high concurrency, with the optimal draft length shrinking as concurrency rises.
Quantization. An NVFP4 checkpoint ships alongside BF16, using the same NVFP4 kernels that power Nemotron 3 Ultra across Blackwell, Hopper, and Ampere. Nvidia's pitch is that one file serves a data center and a desktop equally well.
It runs on your desk
This is the part that will decide whether Lightning gets adopted or admired. A 3B-active MoE is small enough for Jetson, a GeForce RTX 5090, or a DGX Spark, and Nvidia worked with EXO Labs to characterize DGX Spark performance specifically.
Support at launch spans the tools people actually use locally: LM Studio, llama.cpp, Ollama, and Unsloth. For server deployment there are cookbooks for vLLM, SGLang, and TensorRT-LLM. You can try it on build.nvidia.com or via OpenRouter, and pull weights from Hugging Face or ModelScope.
The harness list is the tell for who this is aimed at. Lightning was trained specifically for popular agent harnesses — Nvidia names OpenClaw and Hermes Agent — with support from NemoClaw, Nvidia's open source security and management stack for running always-on agents. Cline, OpenHands, OpenCode, Kilo Code, LangChain, and Factory AI are all in the launch ecosystem.
Open in the way that matters
Nvidia released this under OpenMDW-1.1, with weights, training data, and recipes. That last item is the differentiator. The release includes Nemotron-RL Agentic Terminal Pivot, an open agentic reinforcement learning dataset used to train some of the coding agent capabilities — the kind of asset most labs treat as a trade secret.
Customization paths are documented rather than implied: LoRA or full SFT via NeMo Automodel and NeMo Megatron Bridge, reinforcement learning and environment-based rollouts via NeMo RL and NeMo Gym. Nvidia's argument is that small models fine-tune faster, cheaper, and on far more modest hardware, and that a Lightning-class model post-trained on your tools and policies will beat a generic frontier call on your narrow task.
Nvidia's own framing: "While frontier models may win the headlines, models like Nemotron 3.5 Lightning earn their medals in the trenches."
Read the strategy, not just the model card
None of this is charity. Nvidia sells GPUs, and a world where every agent step hits a hosted frontier API is a world where inference demand concentrates in a handful of buyers. A world where thousands of companies fine-tune and self-host small specialized models is a world with a much broader GPU market.
That commercial logic is why the open release is so generous — and it's reportedly also behind Nemotron 4. The Information reported on August 11 that Nvidia is developing the family with a largest model of at least one trillion parameters, aimed at rivalling the best open models in the world. Final training is not complete and no release date is set, though employees told the outlet it could be ready as early as late fall. Bryan Catanzaro, Nvidia's VP of Applied Deep Learning Research, is among the leaders of the effort.
Treat Nemotron 4 as reporting rather than announcement. Treat Lightning as shipped.
The Bottom Line
Nemotron 3.5 Lightning is not trying to beat Claude or GPT-5.6, and pretending otherwise would miss the point. It's a bet that the economics of agents are decided in the boring 90% of the token budget, not the interesting 10%.
If you are running long-horizon agents and paying frontier prices for ls and JSON reformatting, this is the most concrete cost lever released this month — 86% PinchBench accuracy, 30% faster through 10,000 tasks, and small enough to sit on a workstation. Pair it with Switchyard, measure what fraction of your steps genuinely need a frontier model, and you will probably be unsettled by the answer.
The models that win the headlines and the models that win the invoice are drifting apart. Lightning is built squarely for the second category.
