Can AMD Break the CUDA Moat? AMD Advancing AI 2026

Jul 25, 2026

∙ Paid

When we published our first AMD software article, we gave AMD a 0% chance of closing the gap with Nvidia in AI accelerators. Software was broken, progress was unexciting, and we were the top bug submitter for many months with dozens of AMD engineers triaging our bug reports.

Six months later, in our AMD 2.0 article, we took the non-consensus position of upgrading from 0% chance to a much more meaningful chance at success. We published this opinion when sentiment towards AMD was sitting at rock bottom, and when most of the market thought we were were being far too optimistic towards AMD.

That view was based on our observations that AMD has the leadership that can create change rather than suffer from committee style leadership. Lisa quickly hopped on a calls with us and has since then implemented many of our suggestions. We saw an AMD that had finally recognized the importance of software and had a sense of urgency, moving in the right direction even if the destination was still far off. Since then, the signal has only gotten stronger.

Based on our experience on AMD software stack this year, we update our view again from non-zero percentage chance to now a great chance of success as long as AMD solves the two major risks we outline below. It is important to highlight that just because AMD gains market share, that doesn’t mean that Nvidia will do poorly. The pie is growing rapidly for everyone, and Nvidia will continue to massively grow revenue. AMD poses potential competition to Nvidia on the software front, and Jensen will need to cut bureaucracy and flatten the layers of different required internal stakeholder approvals required for even the simplest of tasks if he wants Nvidia to move faster and defend their lead.

Anthropic has publicly announced that they will deploy 2GW of AMD’s chips, Lisa Su and team have leaned into an agentic oriented engineering culture. Anthropic Head of Compute Tom Brown explained how he used Claude over the weekend with “/goal” to bring-up internal Claude inference stack on AMD hardware as a case in point. We believe that since AMD’s compiler and most of their kernels are open sourced, that they are better positioned for the agentic age besides one major risk that we will discuss later. Three months ago, our Accelerator model noted that Anthropic will be an AMD customer. We also noted this on our socials a week ago.

In 2023, Microsoft dropped AMD after the MI300X due to unreliable Samsung 2023 HBM memory and poor software quality, subsequently skipping both the MI325X and the MI355X. Microsoft has since made an about face and has announced that it will deploy MI455X Helios. We believe that OpenAI will be the main end customer for Azure’s MI455X racks. In a strategy somewhat similar to the Nvidia-Groq deal, AMD is announcing a deal with Cerebras to do PD disagg for ultra-fast interactivity inferencing.

There are two major risks that AMD needs to guard against:

Supply chain checks and engineering first principles analysis and understanding show that AMD’s first AI rack scale system, Helios, is going currently going through a slow rack production ramp given that it is not using a cableless tray design, which the Rubin Oberon rack is now adopting. Furthermore, since AMD has a weak SerDes design, up to 85% of its backplane needs to be retimed, requiring over over 550 Broadcom ethernet retimers per rack. Furthermore, it is running into backplane reliability challenges during rack production ramp hell.

The chief complaint from most AMD engineers internally is that there is a persistent lack of stable GPU clusters for internal software development teams and a lack of stable GPU clusters for automated testing CI. This is blocking AMD’s rate of progress and it is holding AMD back from harnessing the potential upside of AI coding Agents because each AI Agent requires GPUs as well and also requires a testing tool use loop. We will more explain below on our recommendation section.

If AMD can overcome these challenges, we strongly believe that AMD will be well positioned to do well and take market share. This will be helped along as well by the recent stock option based structure whereby AMD gives Meta and OpenAI close to a 105% equity rebate discount using some clever financial engineering. The full rebate triggers a AMD stock reaches the final level of $600 and once OpenAI/Meta buys enough compute. Helios performance per TCO is so great that AMD that cost per million tokens is practically negative cost when combined with this structure! AMD is practically giving away Helios racks and an 5% extra on top of that to an SF-based nonprofit called OpenAI.

In the first part of our article, we go through deep dive into the Helios architecture and the MI455X (gfx1250) instruction set, which is an clone of Hopper SM90’s ISA. We will also discuss the Helios scale-out and scale-up networking architecture. The second part of the article we will further into details of the software stack. For the third part, we will focus on the economics of owning and operating the Helios rack, and also break down the economies of OpenAI/Anthropic/Meta’s equity based rebate discounts and how this structure affects total cost of ownership (TCO).

We are a daily user of ROCm software on MI300X, MI325X, MI355X and are also the #1 bug reporter consistently every quarter! There are a few 10x engineers we want to shout out to that have been working 997 to improve AMD software and have been quickly triaging our bug reports. Many thanks to Hongxia, Chun Fang, HaiShaw, Thomas Wang, Andy Luo, Seungrok, Bill He, Teresa Shan, Parth, Duyi Wang, Gilbert, and many more. Most of AMD’s best 10x engineers are in Shanghai. AMD’s MoRI collective and UMBP KVCache offloading team, AMD’s disaggregated application forward deployed engineering team, and other AMD teams that understand how to do first principles-based inference engineering are all mostly based in Shanghai. A lot of the ROCm software stack’s most important pieces are built in China.

Recommendations to Lisa Su, Mark Papermaster, Sharon Zhou, Anush Elangovan and Vamsi Boppana

There has been good progress over the past year on automated testing to improve software quality, but the pace of progress has not been as aggressive as needed and there continues to lack of sense of urgency with respect to providing enough GPU clusters for automated testing CI and for internal software development teams. It is always too little too late.

Every time we meet with y’all (Vamsi, Anush) every couple of months and occasionally meet with Lisa, we always highlight that CI could be better and indeed we can point to specific examples where CI is heading in correct direction, but the overall strategy needs to be aggressively ramped up. For example, the Kubernetes Inferencing Pollara NIC CI still sits at 0% parity with Nvidia’s ConnectX Nightly CI. Kubernetes is the layer that most inference deployments across the world use. The issue is not with AMD engineering not wanting to add support for it, rather they are being blocked by lack of investment in internal CI capacity. The planned ETA of AMD reaching parity on this front by Advancing AI 2026 was missed due to cluster issues.

On the vLLM side, vLLM gating automated test progress has massively regressed due to AMD cluster infra stability issues this week. AMD’s hardcore engineers were making good progress on vLLM gating over the past couple of weeks to reach the goal of attaining at least 90% parity with CUDA on gating by Advancing AI 2026 until AMD leadership started pulling clusters away from AMD’s internal vLLM team to deploy elsewhere given AMD’s internal capacity crunch that overly relies on backstopped temporary clusters.

Gating/blocking tests mean the tests are of the highest quality because PRs cannot merge unless the test is passing. This prevents bugs from being merged. While AMD’s leadership may distract non-technical folks by showing their non-gating pass rate, gating parity and gating pass rate are the metrics that truly matter.

We hope that AMD’s leadership can re-prioritize giving their internal vLLM team stable clusters and update their philosophy on capacity planning to prevent issues like this in the future so that AMD’s internal hardcore vLLM engineers focus on the work of reaching 90%+ parity with CUDA vLLM on gating and have the tools needed to operate at the same velocity as the AMD internal SGLang team.

The chief complaint from most AMD engineers is that leadership still needs to update their viewpoint towards providing stable CI clusters that don’t just randomly need to be migrated and shifted from one CSP to another CSP.

Moreover, there is a constant lack of GPUs available internally for development. For single node aggregated inferencing, there are enough GPUs to go around internally. But in the age of Distributed Multi-node Inferencing optimizations (wideEP and disaggregated PD), there is nowhere near enough GPUs. Even with the additional 2,000 MI355Xs coming online this month and the 6,000 MI325X/MI355X coming online later this year, total capacity will still not be enough and remains more than an order of magnitude less capacity than the stable long term clusters Nvidia has for internal development. This lack of GPU nodes is a problem that has been getting even worse due to the rise of agentic coding. Previously, each human engineer required a couple of nodes to conduct DI inference software development. But now with agentic coding, each agent requires GPUs to test their code against and each human can have dozens of agents running at the same time and each agent in turn can only dozens of sub agents running at the same time too. Thus, without Distributed Inferencing simulation tools like DynoSim, there will be an even greater shortage of internal GPU capacity internally.

Furthermore, unlike Rubin (SM107), which uses a very similar ISA to Blackwell (SM100), MI455 (gfx1250) is a completely different ISA from MI355 (gfx950) and has completely different codepaths and kernels tha will require testing on both gfx950 and gfx1250. This is another factor that will continue to stress capacity. The big audacious goal was to have MI455X open-source vLLM, SGLang nightly automated CI by Advancing AI 2026, but this timeline was missed. We hear that the new timeline has been delayed untill October 2026 but they are still trying to left-shift it back to an August/September 2026 timeline.

We view this slow ramp of internal software development GPU clusters and automated testing of CI clusters as one of the major risks slowing the pace of improvement of software quality and performance and we hope that AMD leadership revisits their capacity planning strategy going forward.

Part 1: AMD MI455 Silicon & Helios Rack

Source: AMD

AMD has done it again on silicon engineering. The MI455X is the most advanced chip that comes out of a fab on the silicon front. AMD is the first company to ship 2nm datacenter silicon with both the MI455’s compute tiles as well as the Venice CPU being early adopters of N2. All of the competing accelerator platforms are on N3.

AMD continues to lead with respect to on package integration as well. The MI455 package is the largest CoWoS-L module shipping at 5.5x reticle size. AMD is still the only company adopting TSMC’s SoIC-X hybrid bonding, which allows AMD to scale silicon footprint in the z dimension as well as the x and y dimensions. All this comes together to bring the MI455 to a total of 3,470mm2 of logic silicon in the package, by far the most amount of silicon that is being shipped in a single package.

The package layout borrows heavily from the MI355X. 8 N2 ‘XCDs’ are hybrid bonded atop of 2 reticle-sized base dies that contain SRAM, HBM controllers, as well as the compute fabric, which allows for the XCDs to communicate with each other.

One of the major areas where the MI455X departs from the MI30

[truncated for AI cost control]