翻訳待ち：The Sequence Chat - Issue 912: NVIDIA’s Chris Alexiuk Talks About Nemotron, GPUs and Agentic AI

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

We are back with our interview series and a very special guest today! Chris Alexiuk has been helping developers understand and build with NVIDIA’s rapidly expanding AI stack. We discuss the Nemotron model family, the shift from chatbots to long-running agents, model routing and specialization, what it takes to move agentic systems from impressive demos into production, and where the next generation of AI development is headed. Background 1.Can you introduce yourself? You came up through the practitioner side, teaching people to fine-tune and deploy LLMs, before becoming one of the public faces of Nemotron. How does that background shape how you think about open models? I think, honestly, coming from the “open model tinkering” space has allowed me the opportunity to help bring the vibes of that community into how we do things with Nemotron as it relates to “open model tinkerers”. It also has prepared me for the insane velocity at which this space runs. The Main Sequence 2. Nemotron has had a strange arc: Nemotron-4 340B launched as a synthetic data factory, then came Llama Nemotron on someone else's base, the Nemotron-H hybrids, and now Nemotron 3 as a frontier-class open family. Walk us through that evolution. What was deliberate, what was opportunistic, and what quietly died along the way? Nemotron has always been a place for NVIDIA to create awesome open weight technology to both contribute our learnings back to the ecosystem, to create a productive open science feedback loop - as well as ensure we understand the tooling and infrastructure required to make GPUs go BRRRRR at pre-training and RL scale, as well as understand the ways the ecosystem is using and leveraging AI, we need to be in the thick of it. All this to say, it’s been a very deliberate effort. Any amount of good science will have experiments that don’t make it to prime time, and so there’s definitely directions we’ve gone in the last few years that wound up to not pan out yet - but that’s the great thing, we still learned so much along the way. 3. Nemotron 3 ships in three tiers: Nano (~30B, 3B active), Super (~100B, 12B active), Ultra (~500B, 50B active). Those sizes map suspiciously well to a single GPU, a single node, and an NVL72 rack. Are the tiers derived from scaling laws or reverse-engineered from the hardware? And is that even a bad thing? They sure do map well - and yes, we know that a lot of people run models on various levels of hardware; and designing models that are right-sized to them makes a lot of sense to get the best “bang for your buck” at each “scale” of hardware budget. I think, at the end of the day, there is a reality that exists about “where” and “how” people run models - and so you kind of have two choices: Make models at sizes that adhere to some scaling factor - or make models that “fit” onto certain kinds of hardware. I think the latter makes a lot of sense, and was a great pattern leveraged in the Nemotron 3 family of models. Our model cards make it pretty clear what we think the optimal deployment configurations are, as well. So that’s another hint as to how we’re thinking about this. 4. NVIDIA sells GPUs, not tokens, and its biggest customers are the closed frontier labs. Yet Nemotron ships open weights, data, and recipes. Is this classic commoditize-your-complement? What is the honest internal story for why Nemotron exists?. Honestly, NVIDIA was created to solve the biggest challenges, period. A lot of these challenges fall into the “science” and “AI” buckets right now - and in order for those fields to benefit the most, it makes sense for us to participate wholeheartedly in those ecosystems. We’re not trying to compete in the model game - we’re trying to enable tens of thousands of companies, researchers, and developers to engage with and build on top of AI. Also, as recent letters communicated, we think the path forward for safety and security is the scrutiny of all those companies, researchers and developers to help build us all toward a safer future, basically saying: “Open Source is big dope”. Closed models are awesome, by the way, of course - I use them all the time, I just think the future looks a little more like “a little bit of column A, and a little bit of column B”. 5. You go further than almost anyone, releasing pretraining datasets in the tens of trillions of tokens. Meanwhile Qwen and DeepSeek lead the open weights conversation with closed data pipelines. Does open data actually matter in practice, or is it mostly a trust signal? Releasing open weights models is awesome. Releasing open data is more awesome. There’s this reality where you can’t really, truly, audit the model without the data it was trained on. You can get close, and especially through behavioural analysis you can do cool things and learn all kinds of great things about models - but at the end of the day, open data just gives you that extra “feelsgoodman.jpg”, as it relates to understand why models do what they do. That’s why, not to brag, we’ve released 10s of trillions of tokens of sweet sweet data alongside our models (and recipes, and tech reports, and cookbooks, and data curation methods, and more). Also - data is hard and costly to generate. When I say stuff like we want researchers to build on Nemotron, the data effort is a large part of what makes that possible! 6. Nemotron 3 interleaves Mamba-2 layers with sparse MoE and keeps only a handful of attention layers in the whole stack. When you ablate attention away, what breaks first? And how close are we to needing none at all? I think what we’re all discovering is that Attention is not ALL you need, but you do need some. Our teams spent a lot of time learning what the best ratio between SSM/attention was, and some of those learnings land in our technical reports. At the end of the day, attention is incredible for long context recall - and so I’m not certain we’ll go to a world where we both have extremely long context and no attention without more significant architectural changes. 7. Super and Ultra introduce LatentMoE: tokens get compressed into a latent space before hitting the experts, so you can route to roughly 4x more experts at the same cost. What does that actually buy you? And when you look at routing patterns, is expert specialization real, or a story we tell ourselves? LatentMoE does a few things, really well: First, it is great for latency and throughput bound inference - which is great, because those are the types of inference. All those savings go straight back into accuracy as well by letting us use higher top-K. Routing specialization is real, but the human readable version is mostly a version we tell ourselves. 8. Nemotron is quietly becoming multimodal: vision-language models for document intelligence, speech, retrieval, plus adjacent families like Cosmos for physical AI. How do you think about Nemotron beyond text? Does the hybrid Mamba-MoE recipe transfer cleanly to other modalities, or does each one demand its own architecture? Nemotron 3 Nano Omni was our first foray into multimodal Nemotron on top of the current Nemotron 3 backbone - and it worked great! Our vision and speech teams did the heavy lifting to adapt their encoders to the backbone - which was much of the work. I’d recommend reading through the technical report for more details, it’s a really impressive amount of work! 9. You recently launched Nemotron 3.5 Lightning. What’s it for and what innovations have occurred? Nemotron 3.5 Lightning is the "muscle" for AI agents. When you're running long-term agents, you don't want to waste your expensive frontier model on the grunt work like tool calls or subagent delegation. That’s where Lightning comes in. It’s an open 30B MoE model with 3B active parameters, built specifically to handle that high-volume, execution-layer load - fast and cheap. We specifically trained it for the agent harnesses people actually use, so it's not just fast - it’s accurate for the stuff that matters. We also baked in speculative decoding (MTP) and optimized it for deployment on everything from a data center cluster to a local DGX Spark using NVFP4 quantization. Miscellaneous 10. Who is your favorite mathematician or computer scientist, and why? I’m a Ramanujan guy, myself. There’s something incredible about the raw intuition he had in a field that is often remarked upon as being unintuitive. Currently, AI is made out to be just as unintuitive, and I can’t wait to meet our field’s Ramanujan. 11. Give us one prediction for the AI landscape in 2027 that most of our readers would disagree with. It seems obvious to me that model routing is an important stepping stone - but a stepping stone nonetheless - to a more robust, more ergonomic orchestration layer. Routing is great for pipelines, and for many other mission critical - but internal to the agent - processes, but it’s uncomfortable to interact with as an end-user. So my prediction is that “model routing” will evolve rather quickly into the more general “orchestration”.