NVIDIA Nemotron 3.5 Lightning: The AI Agent Workhorse

-->

Nemotron 3.5 Lightning Review: NVIDIA’s Fast AI Agentic Model

India's Most Futuristic AI Conference Is Back – Bigger, Sharper, Bolder

d

:

h

:

m

:

s

Career

GenAI

Prompt Engg

ChatGPT

LLM

Langchain

RAG

AI Agents

Machine Learning

Deep Learning

GenAI Tools

LLMOps

Python

NLP

SQL

AIML Projects

Reading list

How to Become a Data Analyst in 2025: A Complete RoadMap

A Comprehensive Learning Path to Tableau in 2025

A Comprehensive NLP Learning Path 2025

Learning Path to Become a Data Scientist in 2025

Step-by-Step Roadmap to Become a Data Engineer in 2025

A Comprehensive MLOps Learning Path: 2025 Edition

Roadmap to Become an AI Engineer in 2025

A Comprehensive Learning Path to Master Computer Vision in 2025

Best Roadmap to Learn Generative AI in 2025

GenAI Roadmap for Enterprises

Large Language Models Demystified: A Beginner’s Roadmap

Learning Path to Become a Prompt Engineering Specialist

NVIDIA Nemotron 3.5 Lightning: The AI Agent Workhorse

Harsh Mishra Last Updated : 14 Aug, 2026

8 min read

Long-running AI agents often spend most of their time on routine execution rather than difficult reasoning. After making a plan, they may perform hundreds of tool calls, file reads, validations, commands, and formatting steps, so using a frontier reasoning model for every action can become unnecessarily slow and expensive.

NVIDIA’s Nemotron 3.5 Lightning takes a different approach: a fast, efficient model designed for high-volume agent execution. The idea is simple: use the expensive model to think and the fast model to work. In this article, we examine whether that architecture can reduce cost without sacrificing agentic performance.

Table of contents

What is NVIDIA Nemotron 3.5 Lightning?

Why NVIDIA Built an Execution-Focused Model

Architecture Deep Dive

Why Is Nemotron 3.5 Lightning So Fast?

NVIDIA Nemotron 3.5 Lightning Benchmark Results

Nemotron 3.5 Lightning Pricing

How to Access NVIDIA Nemotron 3.5 Lightning

Hands-on: Using Nemotron 3.5 Lightning Through NVIDIA API

Conclusion

Frequently Asked Questions

What is NVIDIA Nemotron 3.5 Lightning?

Furthermore, NVIDIA Nemotron 3.5 Lightning is an open-weight reasoning and instruction model designed primarily for the execution layer of agentic systems.

Its core specifications are:

SpecificationNemotron 3.5 Lightning

Total parameters30B

Active parameters3B

ArchitectureHybrid Mamba-2 + MoE + Attention

Context windowUp to 1M tokens

InputText

OutputText

ReasoningSupported and configurable

Tool callingSupported

QuantizationNVFP4, W4A16 options

Full precision checkpointBF16

Speculative decodingMTP, DSpark, DFlash

Recommended temperature1.0

Recommended top-p0.95

LicenseOpenMDW 1.1

Release dateAugust 11, 2026

NVIDIA’s official NVFP4 model card also lists single-GPU deployment on a DGX Spark GB10 or H100, with support spanning Blackwell, Hopper and Ampere hardware depending on quantization.

The model is primarily intended for English and programming languages, while Spanish, French, German, Italian and Japanese are also officially supported.

This is important because Nemotron 3.5 Lightning should not be evaluated as simply “another 30B model.”

Of course, its intended job is much more specific.

Why NVIDIA Built an Execution-Focused Model

Consider a coding agent.

It may first need to understand a bug and develop a plan. That is a difficult reasoning problem.

But after the plan exists, the agent may need to:

The first step may deserve a frontier model.

Do all the others?

Probably not.

NVIDIA argues that long-running agents spend a substantial portion of their workloads on exactly these high-volume execution operations, such as tool calls, validation and delegation. Using a frontier reasoning model for every execution step increases both cost and latency.

In short, Nemotron 3.5 Lightning is NVIDIA’s answer.

A possible production architecture becomes:

Next, this changes how we should think about model selection.

Instead of asking:

Finally, which single model should power my agent?

the more useful question becomes:

Similarly, which model should handle each type of work inside my agent?

That is the architectural idea behind Lightning.

Architecture Deep Dive

Meanwhile, Nemotron 3.5 Lightning uses one of the more interesting architectures among current smaller agent models.

NVIDIA describes it as a hybrid:

Mamba-2 + Mixture-of-Experts + Selective Attention + Multi-Token Prediction

The combination matters because each component solves a different efficiency problem.

1. Mixture-of-Experts: 30B Parameters, Only 3B Active

Nemotron 3.5 Lightning contains approximately 30 billion total parameters but activates only around 3 billion for each token.

In a dense 30B model, essentially the whole network participates in inference.

In an MoE model:

On the other hand, the router chooses only a small subset of experts.

You therefore retain much of the representational capacity of a larger model while doing computation closer to a significantly smaller model.

That is central to Lightning’s throughput advantage.

Published runtime configuration also exposes 128 routed experts plus a shared expert, with six routed experts selected per token. The configuration contains 52 hidden layers. Its hybrid layer pattern resolves to Mamba, MoE and sparse Attention components rather than using full self-attention at every layer. These are implementation-level configuration details, so developers should verify them against the exact checkpoint and runtime they deploy.

1. Mamba-2 Layers

Traditional Transformers rely heavily on attention.

Attention is extremely powerful, but long sequences become computationally expensive.

Although Mamba is based on state-space modeling and can process sequences more efficiently.

Nevertheless, Nemotron 3.5 Lightning does not abandon attention entirely. Instead, NVIDIA uses Mamba-2 for much of the sequence processing while preserving selected Attention layers where global token interaction remains valuable.

Conceptually:

This hybrid design is particularly relevant for long-context agents.

Instead of paying full attention costs throughout the entire network, the model mixes mechanisms optimized for different jobs.

1. Selective Attention

Attention is still important when tokens must directly compare information across distant parts of the sequence.

That matters for:

long documents

source-code repositories

multi-step tool trajectories

conversation history

retrieved documents

agent memory

Nemotron therefore keeps selected attention layers instead of switching to a pure state-space architecture.

The architectural philosophy is not “Mamba instead of Transformer.”

It is use expensive global attention only where it adds sufficient value.

1. Multi-Token Prediction

Normal autoregressive LLMs learn:

Token 1 → predict Token 2 Token 2 → predict Token 3 Token 3 → predict Token 4

Nemotron 3.5 Lightning includes Multi-Token Prediction, or MTP, layers that learn to predict multiple future tokens during training. Moreover, NVIDIA added a dedicated continued-pretraining stage for these MTP layers.

MTP improves training signals, but it also becomes useful during inference.

Instead of proposing only:

next token

the system can speculate about:

token t+1 token t+2 token t+3 ...

Those candidates can then be verified efficiently.

As a result, this is one of the mechanisms behind Lightning’s high generation throughput.

Why Is Nemotron 3.5 Lightning So Fast?

On the other hand, its speed does not come from one optimization. It is the combination of several.

The best option therefore depends on concurrency.

There is no universally fastest configuration.

MoE Sparsity: 30B parameters provide capacity, but only about 3B are active.

In short, Hybrid Mamba Architecture: Mamba reduces the need to perform full attention across every layer.

Furthermore, NVFP4 Quantization: Lower-precision inference reduces memory and compute requirements.

Instead, Multi-Token Prediction: Several future tokens can be proposed together.

Of course, Speculative Decoding: NVIDIA provides three speculative approaches:

MTP: Integrated directly into the model. NVIDIA recommends it particularly for medium to high concurrency.

In particular, DSpark: A dedicated draft model optimized for DGX Spark and lower-concurrency data-center inference.

As a result, DFlash: An additional draft model that developers can benchmark against MTP and DSpark for their workload.

Source: Nvidia

While Nemotron 3.5 Lightning combines strong intelligence with up to 4x output speed of similar-sized models, placing it on the accuracy-speed Pareto frontier for high-volume agent workloads.

NVIDIA Nemotron 3.5 Lightning Benchmark Results

NVIDIA publishes both BF16 and NVFP4 results across knowledge, reasoning, coding, agents, instruction following and long context.

In fact, the important observation is that quantization does not dramatically collapse model quality.

Here are the official reported results. Benchmark-native units are preserved, so not every value should be interpreted as a percentage.

BenchmarkBF16NVFP4

MMLU Pro81.9481.62

AA-Omniscience17.5016.63

GPQA Diamond, no tools75.4475.57

HLE, text-only, no tools11.7210.47

SciCode32.6031.38

SWE-bench Verified51.5652.80

SWE-bench Multilingual39.3336.47

Terminal-Bench 2.124.5823.46

PinchBench85.3783.43

BrowseComp36.9736.81

τ³-bench Banking9.289.48

GDPval-AA-V2832865

IFBench loose71.8872.88

AA-LCR52.0049.19

Source: Nvidia

Moreover, NVIDIA says these evaluations were run through a consistent NeMo Gym and NeMo Evaluator-based harness and has published benchmark recipes for reproducibility.

In contrast, an interesting result is how close NVFP4 remains to BF16.

Nemotron 3.5 Lightning Pricing

However, pricing is slightly more complicated than a single number because the model is open-weight and available through multiple routes.

The following reflects publicly listed pricing on August 12, 2026.

Running vLLM locally

Access MethodCurrent CostContextBest For

NVIDIA Build APIFree prototype endpoint1MTesting

OpenRouter free routeFree1MQuick experimentation

OpenRouter standard$0.05 input / $0.20 output per 1M tokens262KSimple hosted API

Fireworks serverlessSimilarly, $0.05 input / $0.01 cached / $0.20 output per 1M262KProduction serverless

OllamaNo per-token model feeRuntime dependentLocal/private use

Self-hosted vLLMInfrastructure costUp to 1MEnterprise/self-hosting

Next, NVIDIA currently offers a free API endpoint for prototyping through build.nvidia.com.

Meanwhile, OpenRouter lists both a free Nemotron 3.5 Lightning route with a 1M context and a standard route currently priced at $0.05 per million input tokens and $0.20 per million output tokens. The standard OpenRouter route currently advertises a 262K context rather than the full 1M model capability.

Finally, Fireworks currently lists exactly $0.05 per million input tokens, $0.01 per million cached input tokens and $0.20 per million output tokens, with a 262K serverless context window.

Pricing and context limits can change quickly, particularly during the first weeks after a model release.

How to Access NVIDIA Nemotron 3.5 Lightning

First, at launch, there are already several practical ways to use the model.

Option 1: NVIDIA API

Go to https://build.nvidia.com/ and login or sign up

Click on your profile picture and then API keys.

Generate a new API key.

Now use this API for inference.

Option 2: Ollama

Install Ollama in your system from

Run the following command in terminal to download and run Nemotron 3.5 lightening locally.

ollama run nemotron-3.5-lightning”

Option 3: OpenRouter

You can also use OpenRouter to run this model. Of course, its listed as a Free model on OpenRouter. Instead, grab an API key and start to use it

Hands-on: Using Nemotron 3.5 Lightning Through NVIDIA API

Nevertheless, NVIDIA exposes the model through

[truncated for AI cost control]