COMPUTEX 2026 — NVIDIA Keynote — Extreme Co-Design: Building the AI Factory

[00:00:00] Ladies and gentlemen, welcome back to the Computex Forum. [00:00:03] Today, our sessions will focus on AI Compute, Infrastructure and Development, [00:00:10] Generative AI and Intelligent Content Applications, [00:00:14] Applied AI for Industry Transformation, [00:00:17] and Data Intelligence, Governance and Security. [00:00:22] 我们非常荣幸邀请到首位演讲嘉宾, [00:00:26] NVIDIA Kevin Deerling先生, [00:00:30] 将为我们带来题为 [00:00:33] Extreme Co-Design, Building the AI Factory的精彩分享。 [00:00:38] It is a great honor to welcome our first speaker, [00:00:41] Mr. Kevin Deerling, SVP Networking at NVIDIA. [00:00:46] Mr. Kevin Deerling will share his insights titled [00:00:49] Extreme Co-Design, Building the AI Factory. [00:00:52] Ladies and gentlemen, please welcome Mr. Kevin Deerling. [00:01:00] Good morning, Taiwan. [00:01:05] Good to be here. [00:01:07] I'm Kevin Deerling. [00:01:09] I run our networking business at NVIDIA. [00:01:12] And I'm going to talk about Extreme Co-Design, Building the AI Factory. [00:01:18] And we're hopefully today doing some thinking at scale. [00:01:21] So, first of all, there's three major things happening with AI today and computing. [00:01:31] Three major transformations happening in parallel. [00:01:34] The first has been happening for two decades now. [00:01:38] We are converting from traditional computing, single-threaded computing running on a CPU, [00:01:46] to accelerated computing, where every application is becoming GPU accelerated. [00:01:53] We're rewriting these applications. [00:01:56] We're parallelizing them onto a massive number of GPU cores. [00:02:01] And we're doing this for design, data processing, medicine, simulations, quantum computing, research. [00:02:09] All of that is happening right now, over decades. [00:02:14] But the big thing that has happened now is generative AI. [00:02:18] It's only a little more than three years ago that we first had our chat GPT moment back in [00:02:26] Just a couple of years ago and we're seeing massive acceleration. [00:02:29] So that instead of searching for things, now we can ask questions and chat and generate content, images. [00:02:37] And now we're moving to agentic reasoning. [00:02:41] And I'll talk more about agents in a second because they're incredibly important. [00:02:46] And we're seeing those agents now, just the beginning, deploying into robots and self-driving cars. [00:02:53] And all of this is causing a complete transformation of the way we build data centers. [00:03:02] In the past, the data center was something that cost money, that you would run IT applications on. [00:03:10] And AI factory generates money. [00:03:13] It has to generate tokens very efficiently. [00:03:17] And so we need to completely re-architect the way we're building data centers, moving from cost centers, [00:03:23] to token revenue generators. [00:03:28] So, what is the backdrop of that? [00:03:31] Why do we need accelerated computing? [00:03:33] And what do we mean by it? [00:03:36] It turns out that Moore's law has ended. [00:03:39] Moore's law said that the number of transistors would double every two years. [00:03:45] and I've been in the industry almost four decades and that has been happening. [00:03:50] And if we define it simply as the doubling of the number of transistors, then we can say that we're still, Moore's law is alive. [00:03:58] But Moore's law said more than that we would double the number of transistors. [00:04:04] and it was also a law of physics and economics. [00:04:09] It said that we would double the number of transistors at the same power and at the same cost. [00:04:16] And that has not happened. [00:04:18] That fell off a decade ago, two decades ago almost now, back in 2005. [00:04:23] and what's called Denard scaling is the underpinning, the physics behind Moore's law. [00:04:29] And Denard scaling ended two decades ago. [00:04:33] And so what we need is to re-invent computing. [00:04:38] And we had to re-invent computing to do accelerated computing to stay on track. [00:04:45] and the massive parallelization of GPUs, all of those cores and taking what were single threaded applications and transforming them into parallel applications. [00:04:57] But now GPUs are everywhere in every cloud, in every office, so it makes sense to parallelize every workload. [00:05:05] And the CPUs must be re-architected with a new mission. [00:05:10] And that mission is to keep GPUs fed and keep them busy doing work and producing tokens. [00:05:19] And just as Moore's law has died, at the same time, we're seeing this insane demand for new tokens and AI computing. [00:05:30] We're seeing incredible growth. [00:05:33] If you look here, the model parameters are growing. [00:05:38] Oh, isn't that nice? [00:05:40] Everyone in Taiwan is so friendly. [00:05:41] They gave me a green laser. [00:05:44] The model size are growing 10x parameters for every year. [00:05:49] And we've seen that. [00:05:50] And it's staying on that track. [00:05:52] The models keep getting bigger and bigger and bigger. [00:05:55] At the same time, you can see that we were growing the amount of tokens that we were using. [00:06:01] This is called test time scaling. [00:06:03] We're actually running inferencing and we're processing and generating more and more tokens. [00:06:08] And suddenly there was an inflection point when we hit the first what was called mixture of expert models. [00:06:15] That's only a year and a half ago. [00:06:17] And all of a sudden we had even more tokens being generated as the models, the AI started to reason. [00:06:24] We would generate a hundred times more tokens internally to generate an output token. [00:06:29] And now with agents, we're going to see this grow even faster. [00:06:34] So the number of tokens are going to continue to grow. [00:06:39] Now we have been scrambling at NVIDIA to keep up because what we need to do is all of this is happening. [00:06:45] We need to drive the cost per token down. [00:06:49] The tokens per watt and the tokens per dollar is really what drives us. [00:06:55] And we're doing that through a bunch of different means. [00:06:58] We're optimizing models. [00:07:00] We're changing the precision from FP16 to FP8 to FP4, 4 bits. [00:07:08] We're doing things with the models themselves. [00:07:11] We're moving our architectures, our chips, which I'll talk a lot more about. [00:07:15] So to meet this demand to deliver token efficiently, we've had to rethink the data center entirely. [00:07:22] And we've moved from data centers to AI factories. [00:07:27] So what is an AI factory? [00:07:32] We completely redesign a data center to optimize it across the entire data center stack. [00:07:41] So Jensen bought the networking company that I worked for six years ago now. [00:07:47] And he said, we are going to optimize across the entire data center stack. [00:07:52] And I said, wow, that sounds really cool. [00:07:55] I don't know what it means, but it sounds cool. [00:07:57] And here it is, six years later, we are still doing that. [00:08:01] And we are still optimizing across this entire stack. [00:08:05] And Jensen calls this the, a five layer cake, a five layer cake. [00:08:10] And what's important there, it really, I'm going to talk about this cake. [00:08:14] And we're doing co-design, extreme co-design in each of these layers, starting at the bottom, moving our way up and then going from the top all the way down. [00:08:26] And we're also within layers up and down across the layers. [00:08:31] And energy is the starting point because energy is foundational layer that determines how much work we can do in the AI factory. [00:08:41] The next layer up is chips and GPUs are only part of it. [00:08:45] GPUs are incredibly important, but I'm going to talk about all the chips and how they work together. [00:08:51] The next layer up is infrastructure, because it turns out if you just build chips and you don't think about how they're built into systems and then how those systems are built into racks and then how those racks scale across and scale out between data centers. [00:09:07] You're not going to achieve the tokens per dollar and tokens per watt efficiency that you need. [00:09:13] The next layer up in the stack is models. [00:09:16] And AI models are much more than chatbots. [00:09:19] There are all kinds of models. [00:09:22] We have multimodal models. [00:09:24] We have transformers. [00:09:25] We have diffusion models. [00:09:27] We have models that scale out. [00:09:29] We're doing disaggregated models. [00:09:32] We're doing all kinds of different things. [00:09:34] And we're training specific models for specific use cases. [00:09:38] Architectures that understand languages and images and all kinds of things. [00:09:44] And we need a new layer of infrastructure that is purpose built. [00:09:50] So to connect and the networking between them, the communications between GPUs, CPUs and storage at factory scale. [00:09:59] And the AI applications that we build on top of that is incredibly important. [00:10:05] And we've completely transformed every business. [00:10:08] And now we are seeing AI agents. [00:10:12] Extreme co-design optimizes across this entire AI factory stack. [00:10:17] And we optimize each layer and up and down and between the layers. [00:10:22] So let's start our extreme co-design journey at the bottom with energy and we'll work our way up through these layers. [00:10:30] So an AI factory requires land, power and shell. [00:10:37] We call it land, power and shell. [00:10:39] The shell is the building. [00:10:41] But we start with energy because energy fundamentally limits how much work the factory can do. [00:10:49] So we need to know how much energy we have. [00:10:51] And normally we can't change that. [00:10:53] It takes a long time to deliver energy. [00:10:55] But we have to use the energy that we have efficiently. [00:10:59] We can't afford to waste energy. [00:11:02] And if you look at this graph here. [00:11:04] This is actually the rainfall by month in Taipei City from a couple years ago. [00:11:10] And I'm glad it wasn't this year. [00:11:14] Looks like a typhoon hit or something, a ton of rain. [00:11:17] And then all of a sudden we have another month where there's not much rain. [00:11:20] And it turns out that it makes sense not to just let all that water run into the ocean. [00:11:28] And so it makes sense to build a dam. [00:11:31] And you can see here that this dam, Fetsui dam that's outside of Taipei, stores the water when it rains. [00:11:39] And then it releases the water when we need water. [00:11:43] So it's a more efficient use of a scarce resource. [00:11:47] And it turns out we do the exact same thing in the AI factory. [00:11:53] The workloads in the factory used to be something where it rained exactly the same amount every day, every month. [00:12:04] Cloud workloads are shown here. [00:12:08] Cloud workloads are shown here. [00:12:09] This is a traditional workload. [00:12:11] And when I look at that, they're completely random. [00:12:14] They're all different workloads, different users. [00:12:16] And what it turns out is we have a very flat load, this very, very flat load. [00:12:22] We didn't have to worry about it in the past. [00:12:25] But AI training is completely different. [00:12:28] AI training, all of the GPUs, thousands of them, tens of thousands of them, hundreds of thousands of them are operating like one giant computer. [00:12:37] They're processing a lot of information, they're adjusting their weights, and then they do what's called the collective operation. [00:12:43] They share all of those weights. [00:12:45] They get some more data and then they start running again. [00:12:48] And what you see is the power load over time. [00:12:51] This is the training workload. [00:12:53] This is the power. [00:12:55] And it's just like rainfall. [00:12:57] You get heavy rainfall, we get peaks of current, and then we don't need to use as much. [00:13:02] And so it turns out that that's a big problem. [00:13:06] It's a big problem for delivering the energy. [00:13:08] It's a big problem for designing the data center. [00:13:11] We used to design for that peak demand. [00:13:13] And that told us how many GPUs we could have. [00:13:16] And the energy company that delivered the power didn't like that. [00:13:20] They said, "Hey, you were consuming all this power and then you stopped. [00:13:23] Why did you do that? [00:13:24] You blew up our transformer." [00:13:27] We actually now with what's called DSX that Jensen talked about in his GTC keynote introduced what is really the operating system of the AI factory. [00:13:39] We smooth out the energy demand. [00:13:42] When we have lots of excess energy, when we're not consuming it, we store it. [00:13:46] And then all of a sudden when we have a peak in demand, we can take that energy that we've stored onto the capacitive and the power supply systems and supply it back. [00:13:58] So we flatten out the energy consumption curves. [00:14:01] We work closely with DSX Flex and DSX LPS to shuttle the current between racks. [00:14:09] We work with the power companies so that they say, "Hey, we need you to dial back how much power." [00:14:15] As a result of all of this, with DXS, we can deploy 40% more GPUs in an AI factory per gigawatt of power. [00:14:26] 40% more GPUs means 40% more tokens and 40% more revenues. [00:14:33] So it's enormously important that we've done this co-design up and down the stack at that energy layer to actually make it more efficient. [00:14:45] So let's go up a layer. [00:14:47] Chips. [00:14:48] Some people think still that NVIDIA is a chip company. [00:14:53] We're an infrastructure company and chips is the starting point and chips are important but not enough. [00:15:00] And having one chip is not enough. [00:15:03] Extreme co-design is what brings these chips to life. [00:15:08] We have seven chips that are co-designed together and optimize up and down the five layer cake. [00:15:16] So when we do that, an example of extreme co-design across the layers and up and down the layers. [00:15:23] Rubin GPU is our latest GPU and it is tightly coupled to Vera CPU through a high performance chip to chip cache coherent link. [00:15:35] And we scale that up at the rack level with NBLINK to achieve huge memory bandwidth and communication between GPUs and CPUs within the rack. [00:15:47] And then we scale this out with Spectrum X fabric. [00:15:52] And we use our connect deck and our blue field devices connect into that fabric and scale across the whole data center. [00:16:00] We scale out between racks. [00:16:02] We scale across between data center halls and now even data centers over hundreds of meters and even kilometers. [00:16:10] This extreme co-design delivers 10x more tokens and one tenth the cost per token by optimizing across this stack. [00:16:20] Everybody said that inferencing was going to be easy. [00:16:24] It's incredibly complicated. [00:16:26] The mixture of expert models, the requirements for NVLINK scale up, the requirements for scale out. [00:16:32] And now with agents, we're going to see even more demand and I'll talk about that in just a second. [00:16:38] So designing a single chip is not enough. [00:16:41] Extreme co-design across the entire stack is required to get the most from the AI factory. [00:16:49] So let's move up the stack to infrastructure. [00:16:53] So I said chips are not enough. [00:16:55] So we had to build systems, but even systems are not enough. [00:17:00] We need to coherently connect all of these systems together with our NVLINK, NVLINK 72 in a rack. [00:17:09] And when we did that, we used copper. [00:17:12] Copper where you can. [00:17:15] Jensen says copper where you can. [00:17:17] So the cabling that you see is incredibly elegant in our cable cartridges. [00:17:22] Why do we use copper? [00:17:23] Well, it's very cost effective. [00:17:26] It's reliable and it doesn't use power. [00:17:29] It has lots of good attributes. [00:17:31] The problem is, is over the last 25 years, we used to be able to run at 2.5 gigabit, 30 meters across a copper cable. [00:17:40] As we've gone faster and faster and faster, the distances that we can travel over copper has gotten smaller and smaller and smaller. [00:17:49] Today we're limited to a few meters. [00:17:51] Maybe within a rack we can go up and down. [00:17:54] So we've tried to squeeze more and more GPUs into a rack that we can connect with copper. [00:18:01] And when we do that, we don't do that because we like to make things hot. [00:18:06] That's not why we do that. [00:18:08] We do it because we can only connect a certain amount of distance with the copper. [00:18:12] So we created a different problem, which is the cooling problem and the supply of the current, the DC bus bars that we use. [00:18:20] We used to do 50 volt supplies, 56 volt supplies. [00:18:24] We're moving to 400 and 800 volt supplies. [00:18:27] We're moving to liquid cooling. [00:18:29] That is part of this design that we're doing, the really extreme innovation. [00:18:36] So we have done thermal innovation because moving GPUs close together requires liquid cooling. [00:18:43] And now AI factories have become liquid cooled, DC powered and very energy efficient because we needed to put them together, make 72 GPUs look like one GPU operate as one giant computer. [00:19:00] So now with Vera Rubin NBL 72 and our MGX form factor, we have unified all of this infrastructure. [00:19:10] It is GPUs, the Vera CPU compute and the Vera Bluefield storage, all in this same form factor, this same scalable form factor that fits into the same rack, liquid cooled, DC powered, energy efficient. [00:19:27] And we scale this with, we scale it up with NVLinked and we scale out with Spectrum. [00:19:33] And to reduce the power further, I said copper where you can, we have to use optics to scale across the whole data center between racks. [00:19:44] Now it turns out that even running over copper, a short distance between the ASIC and the front of a transceiver, an optical transceiver. [00:19:53] We start coming out of the switch chip with the Spectrum Ethernet with a beautiful electrical signal and we run it across the PCB to the front of the panel and we make it ugly. [00:20:04] The copper on the PCB makes the signal ugly because it actually degrades the, what's called the signal integrity. [00:20:13] So we have to reproduce the electrical signal and make it beautiful again and then we drive the optics. [00:20:18] But we can do better than that.And the Spectrum X, the Spectrum CPO switch, co-packaged optics is the first co-packaged optics switch chip that is actually putting the optics right next to the ASIC. [00:20:35] So it's beautiful when it comes out of the ASIC, that electrical signal is beautiful and we convert it to a beautiful optical signal inside of the package right next to the silicon. [00:20:45] And when we do that, we save a ton of power.It translates to tens of megawatts of power savings in an AI factory and that power can then be used to generate more tokens.We can use it with GPUs and CPUs and run tools. [00:21:02] And all of these nodes are connected with the Bluefields and the ConnectX SuperNICs at 800 gigabits per second, moving to 1600 gigabits per second now.And that bandwidth is important because agents have arrived. [00:21:19] So what is an agent?An agent is AI eating its own tail. [00:21:27] And instead of humans generating prompts,AI closes the loop and feeds the prompt back to itself.And it turns out humans are slow.AIs are fast.AIs are impatient.AIs are way faster than humans. [00:21:47] So we created this agentic loop where AIs are talking to AIs or talking to AIs.And the large language models think, plan and act.They read documents and use tools. [00:22:02] Vera is a CPU built to run agentic tools.Vera is an AI CPU that plans.It writes programs.It summarizes the results.And it compacts them.And we have great CUDAX libraries that have accelerated these tools.We have the models.Nematron.Open models.And we have the Bluefield DPUs to make everything secure. [00:22:29] these agents are thinking.And thinking needs memory.And thinking needs memory.And agents need memory.It needs short term memory,which is called KB cache.And it needs long term memory,which is files and objects and vector databases.So these agents are actually using memory.And I say memory,this is not DRAM. [00:22:47] This is agentic memory.This is agentic memory,just like we have memories.We have short term memories and long term memories.The agents need a new class of AI storage to support this agentic memory. [00:23:14] So we have developed a reference architecture for a new class of storage for AI agents.And this is called CMX for short term memory,for KB cache.And STX for longer term memory,like files and object and vector databases.So that we can do similarity searches.We store embeddings. [00:23:41] So humans think in terms.Humans think in terms of language.And a large language model converts those words,those tokens,into ones and zeros.And that is called KB cache.And once it has that native language with KB cache,we can actually store it and restore it.And in the middle here,you see what is called pre-fill and decode. [00:24:05] When you read a document or when you receive a prompt,you actually need to read the entire document.And you're thinking about it and you're looking at what's called the attention mechanism.Attention is all you need.That's a large language model.When we do that,we then store the KB cache and run the back to the decoder.We can call a tool.We need to accelerate all of this.The result of this extreme co-design. [00:24:32] What we need is five times faster token throughput and five times better efficiency using these new specialized AI storage architectures.And of course,we need extreme security too.So Bluefield and DOCA,what we call the STX,LPX RAC running.It enforces data access to this,to the pod at line rate.And this is a huge ecosystem of partners. [00:25:00] that we have available here to do this.So let's move up to models. [00:25:05] Models are at the heart of AI.And we accelerate the world's leading models,both open models and AI lab models.And we continue to optimize models as soon as they come out. [00:25:18] Here you see 30x performance improvement on the latest DeepSeq model using Blackwell.And Rubin runs these models even faster.And this is important because LLMs have to get faster and cost less.Because we've seen that agents are impatient.And they eat tokens for breakfast,lunch and dinner.So we've built open Nemetron models that deliver frontier performance. [00:25:45] Jensen showed this in his keynote.And our partners can customize these into proprietary supermodels.So finally applications.We've reached the top of the five layer cake.Applications are the icing on the cake.We've been building these for two decades. [00:26:04] for two decades.Cu2x libraries.Accelerate everything.CuLitho for chip design.Digital solvers.CFD.Scientific.CuOpch for decision optimization.CuDF for data processing.We've accelerated across the entire AI factory data center stack.We've optimized every application and workload for every business.So finally,we couldn't do this without our partners. [00:26:31] We couldn't do this without our partners.Our partners are a critical part of this extreme co-design journey.We love our partners because this is how we scale.And many are right here in Taiwan.And now we have agents.Agents are new workers that plan, reason and act. [00:26:50] every business should use agentic ai because they make your human workers more efficient way more efficient the productivity gains are enormous and the exact same agentic framework this harness is applicable to every business everything benefits from accelerated computing agentic reasoning and extreme co-design robots medical equipment self-driving cars satellites coding [00:27:20] everything will become agentic so i'm happy to be here in taiwan and share this and with all of our partners to help make the ai factory and do this extreme co-design thank you very much