The AI Infrastructure Utility — Wade Vinson, NVIDIA — DCAC Live 2025 Keynote

[00:00:00] . [00:00:12] Coming to the stage for our keynote of DCAC 2025 AI Factories Unlimited, [00:00:18] building the fourth utility. [00:00:20] Coming to the stage is Wade Vinson. [00:00:22] Give him a round of applause. [00:00:24] I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I. [00:00:38] All right. [00:00:42] How'd you like my intro? I like that music. [00:00:46] So we're going to be talking for the next 30 minutes. [00:00:49] We're going to have time for questions at the end about how everybody here, [00:00:53] all the five disciplines associated, we're going to be building the new utility. [00:00:58] That is the AI infrastructure. [00:01:00] So you folks are going to be the, you, you are the folks who are going to be [00:01:04] digging the trenches for putting the, putting the water pipes in, [00:01:08] putting the rails down, all of the electricity. [00:01:12] This is the new utility. [00:01:15] And, you know, I, I picked this image and 10th anniversary. [00:01:20] How exciting, you know, it, the rocket ship has taken off. [00:01:23] I think we all see that. [00:01:24] And for any of you who have, I've never been to a rocket launch, [00:01:27] but come on, we're in Texas. [00:01:28] So I got to show it there. [00:01:30] And think about that feat of engineering. [00:01:33] And you just, you feel the rumble. [00:01:35] And if you ever see it coming out and there's no guarantee it's going to make it. [00:01:38] Right. [00:01:39] But I mean, it's just, it's just such an amazing feat. [00:01:41] And when it first takes off, you're not sure if it's moving. [00:01:44] It barely looks like it's taking off. [00:01:47] But you know what the astronauts are sitting in there going, you know, [00:01:50] so that's where we are right now. [00:01:52] So I'm hoping that entertain, inspire, educate all the folks here. [00:01:58] I'm going to break it into three sections. [00:02:01] Always like to share a little bit of wisdom from Jensen Wong, our CEO, [00:02:06] because he really gets it. [00:02:08] We're going to talk about our technologies, because I think Kirk drew it on the board yesterday, [00:02:13] you know, that are driving a lot of this. [00:02:15] And then how are you going to participate? [00:02:17] What are the tools that we're giving you so that everybody can join in? [00:02:21] Jensen calls it first principles. [00:02:24] It's, you know, simplify the message. [00:02:27] What are we really trying to do? [00:02:30] And get these QR codes. [00:02:33] Tremendous stories. [00:02:34] After he went to the White House, he was interviewed in this thing, [00:02:38] memos to the president. [00:02:39] And it really made me proud to be an American. [00:02:42] You know, it was just like, wow, this guy gets it. [00:02:45] And then after that, he goes to China and he has like a press conference on the street [00:02:50] and lets all these people talk to him. [00:02:53] And it's literally like, he's the reason that the U.S. and China are not in a trade war. [00:02:58] You know, I mean, literally about how we have so many things that they're, you know, [00:03:03] they're the competition, they're not the adversary. [00:03:05] Yeah. [00:03:06] And how we all got to work together. [00:03:07] And then he was at the Milken Institute, you know, with Mike Milken. [00:03:11] And so, you know, talking about that, you know, that's really simple, basic principles [00:03:16] of the business. [00:03:17] And I'm going to share what he talked about at GTC in Paris, but we had the great fortune [00:03:26] about four or five weeks ago at our headquarters. [00:03:29] We had what we called the CDU summit. [00:03:32] We brought together a hundred of the top people designing. [00:03:36] I see many people from the audience there that are designing all of the liquid cooling. [00:03:42] And Jensen talked about the opportunity, but it wasn't just for the cooling people. [00:03:47] He was really talking to all of us. [00:03:49] So when he got off stage, I said, I know what I'm going to say at DCAC. [00:03:55] And so what he said was, oh, these are the people. [00:04:00] These are the people who are going to solve our biggest problem. [00:04:04] And it's good for them because it's a great opportunity. [00:04:06] But what's our biggest problem? [00:04:08] The reality is if we create energy, bring it into a data center, and that doesn't get converted [00:04:16] into a data center, you know, the opportunity is lost. [00:04:20] And every little bit matters. [00:04:22] So it's kind of really cool where the CEO gets up there and starts talking about PUE, [00:04:26] line by line by line. [00:04:28] You know, so we look at transfer in the power chain, distribution, transformation, UPS, generators, [00:04:37] a little bit of power loss, about 5%. [00:04:40] In the heat rejection, if you're lucky, it's 15%. [00:04:44] If you're lucky, it's 15% of the heat rejection. [00:04:47] And then about 10% inside the IT kit itself. [00:04:51] So power conversion, you know, fans or pumps. [00:04:55] And again, in the chip itself. [00:04:56] So, you know, about 30% of what we paid for, what the data center paid for, what all you [00:05:05] folks built, you know, not getting to the chip. [00:05:07] And by the way, that's CEO math. [00:05:10] You know, it's correct, but it's not accurate. [00:05:13] You know, so, you know, everybody's mileage may vary. [00:05:17] But what does this mean? [00:05:19] Every bit of energy that doesn't go to work is huge. [00:05:23] So take one gigawatts of data centers. [00:05:26] Not one data center, but, you know, just in general. [00:05:29] That capital investment, because the chips aren't free, and the software is not free, [00:05:35] and all your time is not free, is about $50 billion. [00:05:38] So think about that opportunity and what that means. [00:05:43] So that big purple slice, that's the 30%. [00:05:47] So NVIDIA spends a whole lot of time on the rest of the pie to optimize. [00:05:51] You know, we bought Mellanox, all the GPUs in green. [00:05:55] You know, we're doing stuff with storage. [00:05:57] And so, you know, these are the technologies that we work on all day, every day to try to optimize it. [00:06:03] We've got a dynamo's the software that kind of moves the power around to the storage. [00:06:08] We've got the high bandwidth interconnect where we put two chips together. [00:06:12] We've got high bandwidth memory, co-package optics. [00:06:15] So these are all the things we're working on. [00:06:17] But what is that opportunity? [00:06:18] What does that mean? [00:06:19] Well, what that means is this inefficiency. [00:06:24] So somebody's investing $50 billion, with a B, $50 billion, and it's reasonable to expect over five years that they want a $200 billion return per gigawatt. [00:06:38] You know, and how many people in this unit? [00:06:41] Yeah, this is big numbers. [00:06:42] And so imagine if 30% of that you don't get. [00:06:47] So you were expecting, you told your investors you were going to return $200 billion on their $50 billion. [00:06:57] I don't know about you, but $60 billion of lost opportunity per gigawatt, that seems like something we all want to go after. [00:07:06] So he suggested, you know, like your opportunity, as you look around, you've got your own little part of the data center, your own company, your own initiative. [00:07:17] You know, you're Nemo sitting in the goldfish bowl. [00:07:20] You know, you're going, that's what's in front of me. [00:07:23] That's what I'm working on. [00:07:24] But you've got to jump out. [00:07:27] You've got to take some risks. [00:07:28] You never know what happens when you jump out of the goldfish bowl. [00:07:31] But there's a good chance you're going to participate in the $100 billion to trillions of dollars of opportunity. [00:07:38] So what are the things that have happened that say, oh, I just started to scratch the surface on the opportunity. [00:07:46] You know, the waste that's inherent today. [00:07:48] So we're not that far into the journey. [00:07:51] The GPU graphics processor that AlexNet used to do image recognition in 2012, you know, we're not that far removed from it. [00:08:02] That was the seminal moment, you know, 30 years in the making, you know. [00:08:07] Overnight, NVIDIA was successful. [00:08:09] But AlexNet started at Perception AI. [00:08:12] And then, you know, a couple years ago, people started hearing about chatbots and then ChatGPT. [00:08:18] We all understand that. [00:08:20] Now we're talking about agentic AI. [00:08:23] You know, we're literally, you know, the enterprise opportunities where every enterprise is out there investing in it for the trillion dollar economy. [00:08:32] And then robots, physical AI. [00:08:34] So the opportunity is just growing. [00:08:37] How does that translate to the folks in this room? [00:08:41] The x-axis of compute is the amount of hardware that our customers buy, i.e., data centers. [00:08:48] And so everybody kind of understands in the how scaling worked great for large language models. [00:08:55] Many of you folks have built multi tens to hundreds of megawatt data centers for training. [00:09:02] But it turns out that that's not good enough. [00:09:05] In order to make these models profitable, you have to go to school. [00:09:09] It's got post-training. [00:09:10] Well, the prediction of the amount of IT that needs to be spent is 30x. [00:09:17] So post-training needs 30x of what the pre-training did. [00:09:21] And then when we get to long thinking, reasoning, test time, it's not you asking a question in a prompt. [00:09:29] It's kicking off multiple models, dozens of models. [00:09:32] They're watching videos. [00:09:33] They're coalescing PDFs without you in the loop. [00:09:37] So hundreds, thousands of queries. [00:09:40] And so that is going to take 100x the amount of IT. [00:09:44] So the good news is the opportunity is growing along that compute axis. [00:09:49] And, you know, we talk about the dynamo and the utility. [00:09:55] And so, you know, Thomas Edison, 1882, took coal, fired machinery to spin a wheel and generated electricity. [00:10:06] And overnight, he made a ton of money. [00:10:09] Not really. [00:10:10] He had 82 customers down on Wall Street. [00:10:15] 400 lamps. [00:10:16] That's how the electrification started. [00:10:19] I think we're past that stage in our business. [00:10:21] But it was not a business model until, because he made money at night when people came home and turned lights on. [00:10:29] But it was actually those same factories started saying, hey, I don't need the coal-fired plant in my factory. [00:10:36] Let me use this new thing called electricity. [00:10:38] I don't know if you've seen pictures of the streets dug up in New York City to lay the wires. [00:10:42] It's just an amazing thing. [00:10:44] But it really took off when the first killer app happened. [00:10:48] And you think about how the world has changed because of refrigeration. [00:10:52] You know, commercial, air conditioning, consumer, it's everywhere. [00:10:56] So that's where we are right now in our new utility. [00:11:01] We're at the stage where we're saying, okay, think about it. [00:11:05] We're like a GE back in 1900, you know, smoking our cigars and wearing suits going, pal, we built this thing. [00:11:11] How do we get customers to use it? [00:11:13] Some bright young intern says, I know. [00:11:15] Let's just have them waste it by burning bread. [00:11:18] And so the most amazing invention to use electricity, and I kind of make a joke. [00:11:22] Jensen made a joke about it. [00:11:23] You go into Walmart or Target, there's 50 different toasters. [00:11:28] Who would have thought that burning bread was so popular? [00:11:31] Not nearly as popular as the hairdryer. [00:11:34] I don't need it as much, but we can all see. [00:11:37] We don't know what the big app is going to be. [00:11:39] We don't know what the killer app is going to be. [00:11:41] But these things are, you know, look at how much of them are everywhere. [00:11:45] So that's the expectation for the AI infrastructure as a utility. [00:11:51] So we've got the big AI factory, you know, and so I think everybody understands the first application. [00:11:57] You know, now everybody, you know, doing ChatGPT, they see the value prop of it. [00:12:01] Heck, you can even enhance it at home with your little Dynamo unit. [00:12:05] But, you know, I don't know that's the killer app. [00:12:09] Not everybody is participating in that. [00:12:11] But now you look at agentic AI. [00:12:14] Everybody, every corporation, every person is on a route to have their own personal Jarvis. [00:12:20] If you haven't watched Iron Man 1, the very first one, you've got to watch it. [00:12:25] I mean, look at what Jarvis does for Tony Stark. [00:12:28] It's part of him. [00:12:29] And so that capability unlocking human potential and business and enterprise and all of that opportunity. [00:12:36] So we know that's there. [00:12:37] You know, maybe it's going to be more successful than the toaster self-driving cars. [00:12:42] We all think maybe they'll be flying someday, but those need tremendous amounts. [00:12:46] They don't drive without artificial intelligence. [00:12:49] And then really, that's just the first instantiation of physical AI in the robot. [00:12:54] So I have one hair dryer in my house. [00:12:56] I don't know if I have any robots in my house, but I bet eventually within a year, [00:13:00] the number of robots in my house will exceed the number of hair dryers. [00:13:04] So huge opportunity. [00:13:06] And it's just beginning. [00:13:08] So if you think of all the data centers in the world, all the data and all the data centers, [00:13:14] it was human beings. [00:13:16] Somebody, oh, that's good. [00:13:17] Oh, let me read that. [00:13:18] Let me write. [00:13:19] Oh, let me enhance that. [00:13:20] Let me add to that. [00:13:21] And so that, you know, and we can suck all that in, you know, automatically today in all [00:13:27] these applications. [00:13:28] But the world was really limited. [00:13:30] The data centers we have today are limited because there's only about 30 million people [00:13:35] who knows C or C++? [00:13:36] Anybody in here a programmer? [00:13:37] I'm a mechanical engineer, not a computer scientist at all. [00:13:41] But there's only about 30 million programmers. [00:13:43] But now, everybody can program in natural language programs. [00:13:48] So you can imagine the number of data centers and the AI is going to need, [00:13:52] it's going to take off like crazy now that more people are programmers. [00:13:56] But you know, humans can only ingest so much into their brain. [00:14:00] They can only type so fast. [00:14:02] And gosh darn it, some of us need to sleep at times. [00:14:06] But just like we think of the robots doing work for us. [00:14:12] The old style of IT was a tool. [00:14:15] It was only useful when you were using it. [00:14:17] You know, that shovel sits on the ground. [00:14:19] Oh, that's pretty cool. [00:14:20] You got to pick it up and do something with it. [00:14:23] The old style of IT was that shovel. [00:14:25] But now, when you think of how a robot can be digging for you, everybody gets that. [00:14:30] Well, now think of digital robots. [00:14:33] So now, all this data. [00:14:36] They don't sleep. [00:14:37] It's kind of scary sometimes. [00:14:39] They're sharing all the data back and forth, all that data creation. [00:14:44] So again, the opportunity is huge. [00:14:47] So we need everybody to lean in. [00:14:51] My first 10, 12 minutes here, everybody lean in. [00:14:54] Because when you jump out of that bowl and you participate in each one gigawatt of opportunity, [00:15:00] you might just be the biggest animal on the planet someday. [00:15:03] So we encourage everyone to lean in on that. [00:15:06] So let me pivot over to what we're doing, because I think everybody says, [00:15:09] "Oh, I got to hear what Nvidia's doing." [00:15:11] I'm not going to announce anything new, but I'm going to tell you about some things that we just announced. [00:15:15] So we're not a chip company. [00:15:17] We're not even a server company. [00:15:20] We're really not even a rack company. [00:15:22] We're an AI factory. [00:15:24] Right? [00:15:25] You know, I'm an AI factory. [00:15:26] I'm a walking factory. [00:15:27] Yeah, I guess. [00:15:28] I'm alive. [00:15:29] I'm not data at all. [00:15:31] But that is how we think about the problem. [00:15:35] And so it's one... [00:15:38] It's not just... [00:15:39] The thing is, the GPU is a whole rack. [00:15:41] That whole system. [00:15:42] But each... [00:15:43] That's an AI factory by itself. [00:15:45] So it doesn't have to be this giant gigawatt thing. [00:15:48] Each thing matters. [00:15:49] And we're just getting started. [00:15:52] So last week, we announced something new. [00:15:55] And I wanted to share this with you in case you missed it at the AI Infrastructure Summit. [00:15:59] It's on YouTube. [00:16:00] Inference. [00:16:01] That's where that's where the money's going to be made. [00:16:03] Right? [00:16:04] Oh, it's all different. [00:16:05] But you think about, people are using their GB200 racks. [00:16:09] And when they look, when they say, "Well, you know, it's really great at harvesting data, but it's a two-part problem." [00:16:17] One part, they call it context pre-fill, they're generating values, or they're adding it in, and then they're generating the values for it. [00:16:24] And so the output is, you know, one GPU gets one output. [00:16:28] Many of our top customers have started to carve up their GPUs. [00:16:33] And so they said, "Wow, by putting a second GPU on to do some of that, this key value cash transfer, that they're using two GPUs, and they're getting six times the throughput." [00:16:46] So we're at NVIDIA, and we innovate, and we say, "Gosh, they're using their really expensive GPU for that. Let's do something else." [00:16:53] So we came out with CPX, Rubin, Vera Rubin, a new GPU. [00:17:00] And so in the same rack that the GPUs were being delivered in today, you can see the Rubin CPX chip. [00:17:07] And really, the market for this is, we're just scratching the surface, but the people who are rendering video need 100,000 tokens in a query. [00:17:19] That's what it's about. [00:17:21] Anybody doing large software, using software to program 100,000 million token models. [00:17:27] And so that's where we developed it. [00:17:29] People are using it. [00:17:30] But our estimate is that it's a 50X improvement. [00:17:35] So instead of taking your expensive GPU and slicing it up, now you get a 50X return on your investment by putting the Rubin CPX processor in it. [00:17:48] Now, that does crank up the rack power a little bit, and some people are rack limited. [00:17:52] And you don't always know what your inference ratio is. [00:17:56] So we have two of them. [00:17:58] So you can get a whole rack of the Vera Rubin CPX, and it can be at the ratio of it, or you can put it in the Vera Rubin tray. [00:18:06] And so I just, a little bit of tidbit, again, not a computer scientist, but I figured, you know, we're continuing to invest. [00:18:14] Jen said, I showed this, you know, every year we're coming out with something new. [00:18:18] Blackwell, Blackwell Ultra. [00:18:20] Rubin, you see the Rubin CPX in there. [00:18:22] Rubin Ultra. [00:18:23] We talked about Kyber, where we're literally going to put 576 connections into one rack. [00:18:31] You know, and then Feynman, so every year we're coming out with something new, so the opportunity is going to continue to grow. [00:18:37] And what I wanted to do here is, I think a lot of people have trouble understanding it. [00:18:42] So my friend Nicole here in Austin decided to bring me a little prop to help understand how this thing really works. [00:18:54] So I talked about Tony Stark and Iron Man, but Jensen brought this out to the Consumer Electronics Show in Vegas. [00:19:03] You know, he did a superhero. [00:19:05] He had great music. [00:19:06] But this is the scale. [00:19:08] You like that? [00:19:10] Oh, I like it. [00:19:11] I like it. [00:19:12] For those of you who miss Vegas, now, oh, you wasn't at Yotta. [00:19:14] Now you can feel like you're Yotta. [00:19:16] And I just thought this was cool. [00:19:19] So we're going to have this up. [00:19:20] You can take pictures with it. [00:19:21] Afterwards, we'll come talk about it. [00:19:23] You know, I don't have special music. [00:19:25] And we're not done. [00:19:27] We talked about the 576. [00:19:30] Hey, so where's the forklift? [00:19:32] No. [00:19:33] It's the same size. [00:19:35] Literally, we're taking four of these, cramming them together to put that into the rack. [00:19:42] So long story short is we're continuing to invest. [00:19:46] And I was able to bring the toy out. [00:19:48] So thanks. [00:19:49] Here you're going to call. [00:19:50] Thank you. [00:19:51] I don't want to know what would happen to me if she broke it. [00:19:54] So, how do you participate? [00:19:59] What are we giving you here today? [00:20:03] All five phases to participate. [00:20:05] And it starts with what we did as we moved up to the data center scale. [00:20:11] We said it had to go into every data center in the world. [00:20:13] It couldn't be a special data center. [00:20:15] You know, I come from the world of HPC. [00:20:17] So I built very custom specialized data centers for the US government as part of my career. [00:20:22] But that was not, that just doesn't scale. [00:20:25] So all of ChatGPT, everything for agentic AI kind of started with the H100 racks that could go into every data center in the world. [00:20:34] You know, 500 GPUs on a common hot aisle containment, about 650 kilowatts. [00:20:40] So when we started the SuperPod, the AI factory, a year and a half ago, we wanted to make sure it could go into the same footprint. [00:20:50] So 576 GPUs, and you've probably seen our data. [00:20:55] For the AI factory, this gets 50 times the performance of Hopper. [00:21:01] The number I'm giving there, 1.7 megawatts, when we laid it out with the 60 amp connectors, we were kind of forward projecting. [00:21:11] So this technology gets you up to about 200 kilowatts a rack. [00:21:14] You know, so into the Vera Rubin timeframe. [00:21:17] And when we say you can put it everywhere, we really mean it. [00:21:21] So that's a 3D model we have. [00:21:23] So we're going to be at OCP. [00:21:24] Where's Rob? [00:21:25] There's my first OCP plug. [00:21:26] Well, there's OCP everywhere on here. [00:21:28] So at our OpenUSD, so we're modeling it. [00:21:32] So everybody can get those models, can enhance them, can build them in their own factories. [00:21:37] And you take the 100 gigawatts of aging data centers that have old CPUs in them, or on-prem data centers, or universities, or national labs. [00:21:53] Take out the old aging IT, and with our model, you can put it in here. [00:21:59] So it's not a brown field. [00:22:01] It's a gold field. [00:22:02] You're turning an old asset into something. [00:22:05] And everyone can participate. [00:22:07] You've already got these at Equinix, at QTS, at Digital Realty, and I'm sure many of the other data centers are putting them in. [00:22:16] You can buy these modules, all the IT from all of our partners, HP, Dell, Lenovo, Supermicro, putting them all in. [00:22:24] So what's our future-proofing? [00:22:26] Everybody says, "Oh, wow, that's great, but my customer's 18 months away. [00:22:31] And I don't have time to retrofit. [00:22:34] What can I do today?" [00:22:35] And so the new piece of information given today is we're going to go up to the 100-amp connector. [00:22:43] It's easy to get. [00:22:44] So instead of the 60-amp connector, we're going up to the 100-amp. [00:22:48] And we're going to true three-phase, so not pulling a neutral in. [00:22:53] And when we do that, and looking at sizing, and this is your choice of how you size the infrastructure around it based on your generators and your chillers and your UPSs. [00:23:03] But we think you bring in these eight 800-amp feeders. [00:23:07] We use six of them. [00:23:08] And that technology gets about 400 kilowatts a rack. [00:23:13] I'm not making an announcement about our power level. [00:23:15] I'm just saying the technology that we're giving everybody here, the, what was it, the 3% for site prep and 3% for the engineers who are designing it, the people doing operations, commissioning. [00:23:29] You can use this model, put your VR glasses on and train your whole workforce with this model. [00:23:36] And we're going to be talking about that as part of our digital twin sessions at OCP coming up in a couple of weeks. [00:23:43] But when I say support equipment, this is really, you know, I showed the shield kind of fun, but, you know, there's some support around it. [00:23:51] You need, you need, you know, the, you need the Ethernet, you know, Spectrum X or InfiniBand on this. [00:23:56] You need the, you need the north-south connections here. [00:23:58] You know, you got to make sure you get the power and water all in there, but that's the support stuff around it. [00:24:04] And so we think kind of the new number for this super pod that can go anywhere is about three and a half megawatts. [00:24:12] And you saw the roadmap, you saw how power is going up over time. [00:24:16] And so again, in the future, you probably have to talk about 800 volts DC. [00:24:21] We're going to be at OCP talking a lot more about that, but it's easy to take two sets of racks power and run it into one. [00:24:32] Actually, you can see it on my shirt here, and networking. [00:24:35] And we might use that adjacent rack to have some batteries in it, some energy conversion. [00:24:40] Because again, this is meant to go everywhere. [00:24:41] You're not going to build a brand new data center for this. [00:24:43] Use the data centers that you've got today for this. [00:24:46] And there's my proof point. [00:24:49] So this is in the Equinix data center. [00:24:51] And so you can take two sets of feeds and run it into one. [00:24:54] You can take two sets of networking. [00:24:56] And if you look at the power and the water on the back of the rack, there's already a second water tap. [00:25:01] So we think this rack is 600, the OCP ORV3 that we've donated is MGX. [00:25:07] You know, we think it's about 800 kilowatts capable. [00:25:11] And then you heard us talk about Kyber, you know, the 576 GPUs in one rack. [00:25:17] Well, I got a shim in between my existing data center, but I didn't change my data center. [00:25:23] I was able to put two 1.6 megawatt racks, if such a thing ever exists, into an existing data center. [00:25:30] You notice the CDUs are now gone because I kind of think we integrate them into the plumbing connections on the rack. [00:25:37] And so you started with the air-cooled old data center. [00:25:41] Stick with us. [00:25:42] Let's work together. [00:25:43] We don't think we're leaving anybody behind. [00:25:45] These models are available. [00:25:47] We're super excited about it. [00:25:49] Now, there's another class of customers. [00:25:52] And I'll start wrapping up here as we talk about them. [00:25:55] And we call these the NVIDIA cloud providers or cloud partners, you know, who don't want just one SuperPi. [00:26:02] You know, "Well, I think I need 8,000 for my workload," or "I think I need 18,000." [00:26:06] So we're using these same parts to build out these very dense GPU racks. [00:26:11] And again, same number. [00:26:13] You can build out in that same 3.5 megawatts this module. [00:26:17] And you just repeat it forever. [00:26:19] Now, I will say that our 512 port switch, you can't really see it, but it would be over here. [00:26:27] The 512 port switch is going to really want us to make a slightly longer row. [00:26:33] And once we think we get to that longer row, we start and can we get all that stuff into it? [00:26:40] But we think we can. [00:26:42] And so that, you know, kind of this technology working together gets us up to about 216 kilowatts of racks. [00:26:48] But going to the limits, 1600 amp, right? [00:26:53] We think that the world can build 1600 amp circuits, I think, you know, and we can bring those in. [00:26:59] So, you know, am I going to say that's going to last forever? [00:27:03] But when you look at how we show with the density of the technology and everything that, you know, kind of in that 6 megawatt range that we can do it. [00:27:11] Now, when you have these modules, though, you don't see a lot of the other support racks in them. [00:27:17] And of course, we can double the powers we set over time and we can put Kyber in it. [00:27:23] And so these same modules we use to build the support. [00:27:27] So we've got our version of the OCP rack that actually can do higher power switches, even with liquid cooled switches. [00:27:34] And so you can use the same module for that when you've got a medium density. [00:27:39] And it even works for racks that are just rack-based PDUs, right? [00:27:43] It's got all the containment in it. [00:27:45] And this isn't our product. [00:27:46] We literally just designed this and we're giving it to the world. [00:27:49] And your favorite vendors can build modular data centers out of it, can build smart runs, infrastructures, ecostructures. [00:27:55] I'm missing somebody's brand name of their modules. [00:27:58] And we're trying to standardize this. [00:28:00] I think Kirk was talking about the car industry, right? [00:28:03] You know, Ford and the Dodge brothers and everybody getting together. [00:28:06] It's NVIDIA's responsibility to do this. [00:28:09] We're defining this. [00:28:10] And while, you know, we're doing it because we want it to be able to go into any data center anywhere. [00:28:19] And so this just happens to be a data center in Northern Virginia that I'm a tenant in. [00:28:23] And you go, "Oh, that looks like a 200 megawatt data center." [00:28:26] But it's really not. [00:28:28] It's really just a collection of eight megawatt data centers. [00:28:31] It's actually 16 because there's two, you know, eight megawatt floors. [00:28:35] But as an AI factory, I want to take all the power 8766 hours a year. [00:28:42] And I want to, and I got leap here and there, that can convert to tokens. [00:28:47] When the power is available, I want to put GPUs in and turn tokens. [00:28:50] When it's not the hottest day of the year, I want to convert that into tokens. [00:28:53] We're trying to take that $50 billion investment per gigawatt, recover that lost $60 billion. [00:29:03] And so this is the models that we built. [00:29:05] So yes, you can fill up one of these data centers with our modules, with mail servers, with logon servers. [00:29:13] And we might need that. [00:29:16] We might need a whole bunch of support, you know, for each 200 megawatts of servers. [00:29:21] I might need a lot of support. [00:29:22] So the modules are good for low density, really great for high density, and ultra high density. [00:29:29] In that data center, we would take the two stories and run the power from the second floor into the first. [00:29:34] So literally, you know, getting these racks up to 400 kilowatts a rack. [00:29:39] Split density, because you don't want to waste an asset. [00:29:41] Stranding power, kiss of death, right? [00:29:43] So you want to be able to mix and match and move things around in the data center. [00:29:47] And here's my favorite, this is for Nicole. [00:29:50] And so we're going to do this ourselves. [00:29:52] So in our own data centers, instead of waiting for the fit out, we're pre-populating these modules. [00:29:59] And if it turns out to be high density, we drop the CDU in, in two of the rows, but not all of them. [00:30:05] We've already pre-commissioned the water in some of those rows. [00:30:08] You move the power around. [00:30:09] So we think we really got a cool story here. [00:30:12] Because we want the whole industry, as I wrap up here, to participate. [00:30:16] So we want you to buy from NVIDIA, but if you've got a bunch of Intel or AMD or other, [00:30:22] look, there's nothing, there's nothing proprietary about this module here. [00:30:26] You, you as inventors, you make it your own thing. [00:30:28] You make it proprietary. [00:30:29] You sell your special sauce in it that's easier, faster, it takes less people to commission it. [00:30:34] It's safer to commission. [00:30:36] It's cheaper. [00:30:37] And so, I'm going to come back to the business slide as I end. [00:30:42] When you think about the investment, we talked, Ian talked about this last week. [00:30:47] If someone gave you a free GPU, maybe Bill's got a friend who's going to give a free GPU. [00:30:54] That's one quarter, or Dean, one quarter of the power. [00:30:58] But you still had to put the data center around it. [00:31:02] Compared to our $3 million rack, is that that free GPU will generate $8 million in revenue. [00:31:10] Whereas our $3 million GPU will generate $30 million. [00:31:14] So free is not good enough. [00:31:16] So we get a 10x investment. [00:31:18] And again, remember I just told about CPX, a 50x investment. [00:31:22] So we're going to keep going, working together. [00:31:25] The 30% we're helping with now as well. [00:31:29] Just last week, we announced that we've got a gigascale reference design with many of the partners there, so that everyone can benefit. [00:31:39] We're saying this is how we would do it if we were building an AI factory with some of our biggest customers, the hyperscalers who are building the models. [00:31:47] And remember I said, well, there's no such thing as a one gigawatt data center. [00:31:53] It's a bunch of small data centers. [00:31:55] If you peel the roof back and you look inside, it really is those same modules. [00:31:59] It really is just a bunch of 10 megawatt things, ka-chung, ka-chung, ka-chung, ka-chung. [00:32:05] And who are the commissioning people and all the labor people? [00:32:08] Boy, you're going to be, you know, how do you get all that without tripping over each other? [00:32:12] You know, there's so much opportunity for innovation in that. [00:32:16] And again, I talked about my shirt a little bit. [00:32:19] It's the gold field. [00:32:21] This is the new AI infrastructure. [00:32:23] And what I wanted to do is because we're here at DCAC Live in the 10th anniversary, we're going to release a special software. [00:32:34] So if you have one of these in your factory, and for those of you who missed out on Vegas, so that you can show all your customers what your favorite conference that you really want to attend is all about. [00:32:48] By the way, it doesn't really work like that. [00:32:51] But I figured that would be a good time for me to open it up for some questions. [00:32:55] Again, for those of you who missed Vegas, you know, there's your flashy, flashy lights there. [00:33:00] Okay. [00:33:01] How are we doing? [00:33:02] How was I? [00:33:09] Oh, I get this all the time. [00:33:11] Now, a data center, it's got mail and storage and curated data. [00:33:19] And it's a cost center for a company. [00:33:22] And it's got its levels of resiliency. [00:33:25] Two N plus one. [00:33:26] Is Peter Gross here? [00:33:27] I always say, it's a Peter Gross special. [00:33:29] And I love Peter. [00:33:30] You don't get fired for building a great data center for data that never goes down. [00:33:35] An AI factory is a revenue producing factory. [00:33:40] It's different technology. [00:33:42] The CPU itself is that you are making information in it. [00:33:46] You are making money in it. [00:33:48] And you want all those tokens being hit as hard as possible. [00:33:52] How can we take GPU, total design power, heat flux, before even cold plates? [00:33:59] Oh. [00:34:00] So, we think the right answer is two water temperatures. [00:34:05] Our chips, even through Vera Rubin, take 45C. [00:34:10] So, we're inventing new thermal interface materials, new cold plate technologies. [00:34:16] All of that information exists. [00:34:18] So, we believe that as far into the future as we can predict, that we want to be chiller-less cooling with 45C. [00:34:25] And if it turns out that we get more performance if we run a little bit colder, okay, we will. [00:34:31] But it's still the same grid to token conversion efficiency. [00:34:35] How many tokens can I get for what's actually on site? [00:34:38] I don't know if anybody parsed some of these questions. [00:34:43] Standardizing direct-to-chip or emergent. [00:34:48] We've already standardized on direct-to-chip. [00:34:51] You know, I would say a GTC last year when Jensen got up on stage, you know, a year and a half ago, [00:34:56] he stopped talking about the 2x36. [00:34:59] So, you might see like the Mount Diablo MetaRack, you know, that's got these giant winged liquid-to-air heat exchangers that go into air-cooled data centers. [00:35:08] Great product. [00:35:09] Gets all the benefits of the performance, but it went into an existing air-cooled data center. [00:35:15] But liquid cooling is here. [00:35:17] Right. [00:35:18] All right. [00:35:19] I think I got nine minutes. [00:35:21] So, any of you got microphones coming around? [00:35:23] I'm not going to be shy. [00:35:24] I've got a microphone here. [00:35:26] If anyone has questions, throw your hand up. [00:35:28] I see someone there in the front row. [00:35:29] I'm coming up to you. [00:35:31] You're on your right. [00:35:34] All right. [00:35:35] Doug Mouchen, going to ask my heart. [00:35:36] All right, Wade. [00:35:37] Always good to see you. [00:35:38] I thought that was a Southeast Asian-like tribal shirt or something you're wearing. [00:35:43] The GPU design. [00:35:44] It looks amazing. [00:35:45] I'm sorry. [00:35:46] Hey, the conversation we had the other day, you know, we said, Wade, these AI workloads, they change in a millisecond. [00:35:56] They're choppy. [00:35:58] You said, Doug, we got something for that. [00:36:00] We've actually synchronized our workloads to even out the draw against the power good. [00:36:05] Can you talk about that a little bit with the team? [00:36:07] Let's see. [00:36:08] What happened there? [00:36:09] My slides are... [00:36:10] We announced new power supply technologies with aluminum electrolytic capacitors in the GB200, GB300 power supplies. [00:36:21] So we have some joules on board that charge when power is down during the communication cycle and discharge to go against the grid. [00:36:31] And then we've got software with it as well. [00:36:33] And so that is integral to our GB300. [00:36:36] And then at the Vera Rubin timeframe, we've got new energy technology. [00:36:41] Matter of fact, we just did a white paper with OpenAI, Microsoft, Amazon. [00:36:47] It's a technical white paper that describes all the different attributes people can have. [00:36:53] We're going to talk about that at OCP Summit. [00:36:55] We're going to have a new technical white paper coming out. [00:36:59] You know, I've said this before. [00:37:01] You know, I think five years from now when we tell our kids what a UPS is, oh, that's the brown delivery truck. [00:37:09] We do power quality. [00:37:11] That's important devices. [00:37:12] We have energy. [00:37:13] UPS vendors are going, ah. [00:37:15] It's a different device. [00:37:16] We do energy storage local. [00:37:18] You do BESS to protect the grid because, you know, generators can't handle that. [00:37:23] But again, it's that 60 billion per gigalot that we're trying to save. [00:37:29] Good question, Doug. [00:37:30] What else? [00:37:31] Anybody upstairs? [00:37:32] Second floor? [00:37:33] Look for hands. [00:37:34] If you're upstairs, feel free to shout. [00:37:38] I'm not a football player. [00:37:39] I can't toss you the mic. [00:37:40] Any hands? [00:37:41] I'm looking around. [00:37:42] I do want to say this. [00:37:43] Kirk and I were talking, and David, and a hand, special made shirt, sublimation printing, [00:37:52] one of a kind. [00:37:53] There's actually two of a kind. [00:37:55] I built another one. [00:37:56] We're going to use it to donate for the veterans. [00:38:01] So put your big bids in for it. [00:38:03] And if you want it today, you can take it home. [00:38:05] But if you wait, I will get Jensen to sign it with his ink pen on the back of it. [00:38:12] So all that money going to the good cause. [00:38:15] So there'll be a second one. [00:38:17] Now, I can't guarantee that, you know, the company won't make thousands of these because [00:38:21] they see me on stage with it. [00:38:23] They weren't sure when I told them about it. [00:38:25] But we think it's, we want to give that to the great cause here. [00:38:30] All right. [00:38:31] I have a question right here. [00:38:32] Question. [00:38:33] Yes, sir. [00:38:34] I'm Craig Russell with Alamo One Environmental out of San Antonio. [00:38:37] Very new. [00:38:38] Just walked in here yesterday. [00:38:39] So on the environmental side, I have a friend that just started with Microsoft Southwest of [00:38:45] town here. [00:38:46] And as an environmental, and he walked in and didn't even have any environmental people in [00:38:52] there. [00:38:53] What would you say on the environmental side? [00:38:55] You know, I'm interested in the cooling capacity you hit on that. [00:38:58] What I've read is closed loop mineral oil versus water. [00:39:02] What's happening on it? [00:39:03] And is there where we're at? [00:39:05] If I understand the question right, a scientist at University of California, Riverside took [00:39:12] a Microsoft paper from four years ago and said, "Oh, look at all this water we're wasting. [00:39:18] But now we're all doing closed loop. [00:39:20] So we fill it up once and then we don't use it again. [00:39:25] And around the world, because we use 45C, we can put 60, 65C out, greenhouses. [00:39:34] District heating in Europe. [00:39:38] Processed plants." [00:39:39] So, and it's not, I would love to say NVIDIA is doing it because we're green. [00:39:45] That's one of our colors, green. [00:39:47] But we're doing it because we're trying to save the 60 billion. [00:39:50] It's about business. [00:39:52] Every, I paid once to make that power. [00:39:54] I want to get as many tokens out of it as possible. [00:39:57] So, you know, that's why the environment matters to us. [00:40:02] Because we generated the energy once and we want to get it all in a token. [00:40:06] So hopefully I answered the question. [00:40:08] Thank you. [00:40:09] Thank you. [00:40:10] Four and a half minutes. [00:40:13] All right. [00:40:14] I have another question right here. [00:40:15] Hit you next. [00:40:16] Hi. [00:40:17] Right here. [00:40:18] How you doing? [00:40:19] Can you speak a little bit about the role of generative AI and how it plays and how it has the ability to optimize energy in the data center as well as using workloads to sort of right size that energy draw? [00:40:32] So I did tell you I'm not a computer scientist. [00:40:35] But that last slide I showed, Ian Buck talked about it in our partnership, Phaedra from, you know, ex-Google folks who are actually using it to look at all the fan walls, all the controllers and all that. [00:40:48] I think they, you know, they're showing that, you know, their least amount of energy saving using this AI agent. [00:40:54] And I watched Jim Gao Monday do a demo and he's like talking to his BMS, you know, in real live questions and asking it why the valve is moving. [00:41:05] But I think they, they, they, they, anywhere from 17 to like 33%. [00:41:09] So, uh, one of our partners, Phaedra on the data center side, Emerald AI. [00:41:14] So the grid connections that you can't get cause it takes to do a study. [00:41:18] So working with, uh, EPRI and the DC flex program as part of it, part of our reason we're doing the, the, uh, gigascale factories is so that we can get these grid connection to it using AI. [00:41:30] Because, you know, the, the, the grid, you know, uh, somebody told me that, uh, one state in the, in the country, someone asked for a gigawatt and they said, can you take 1.2? [00:41:43] How many other states would say that, right? [00:41:45] So it's different regions have different needs and they know their grid. [00:41:49] And, and someone else had told me and said, Hey, we saw this thing in Northern Virginia where the whole grid is pulsing every five milliseconds. [00:41:57] And so my, my comment was, okay, you should know exactly where that's coming from. [00:42:03] And I'm like, don't, don't look at me. [00:42:05] Right. [00:42:06] I mean, we got all the AI tools. [00:42:07] We know where everything is. [00:42:08] You know, we, we, we know the technologies on there and they're like, Oh yeah, that's a good point. [00:42:12] So, so we're, so again, to those two partners there, but instrumenting everything. [00:42:16] And then the AI agents are just going to get better over time. [00:42:20] They're in the toaster hair dryer phase, which are very useful, but they're going to get into the robot phase very quickly. [00:42:28] All right. [00:42:29] Here in the center, we still have another question. [00:42:32] Thank you. [00:42:33] Good morning, Wade. [00:42:34] I'm Kathleen Margolis with Black and Veatch. [00:42:36] I have a question about that robotic phase for you. [00:42:39] I'm kind of going out to the future, like the big blue whale you're using in your imagery. [00:42:43] I'm curious, you're crowdsourced the data center community so that we can help you with the infrastructure to stand up the chips. [00:42:52] And so as we move forward, you have a pathway and I think we're all going to follow. [00:42:58] Um, I'm very curious, is Nvidia already launching in the robotics? [00:43:04] And can you give us a timeline for what? [00:43:07] So there's, there's, I think the question was, it's going to be hard to build all this at scale. [00:43:12] And, and what technologies are we doing to help do it? [00:43:15] And so it's a, it's a multi-phase approach. [00:43:18] We generally, we have our own models. [00:43:20] We're using them ourselves. [00:43:21] We're donating them through open USD step one. [00:43:25] Cadence. [00:43:26] Now you can do CFD in it. [00:43:28] ETAP. [00:43:29] You can do electrical. [00:43:30] So, so these are physics-based models. [00:43:32] Today, they're just 3D models. [00:43:34] These are physics-based models for, and again, we're going to talk about it at OCP. [00:43:39] Now that it all exists in software, then we'll, then, then we'll send the, the, the, the, [00:43:45] the software robots in and they'll manipulate it and train in the real life model, but before [00:43:52] we put them in the data center, like the robots that were here last night, scaring everybody [00:43:56] to death. [00:43:57] They're kind of cool, but they'll, the, the robots will know exactly what to do to build [00:44:02] it at scale in the factories and the data centers before it comes out to the real world. [00:44:06] Does that answer your question? [00:44:07] I wasn't sure. [00:44:08] Okay. [00:44:09] All right. [00:44:10] Down here to your right. [00:44:12] Hi. [00:44:13] This has been fantastic. [00:44:15] If you look back over the last five years and you think about the average data center build [00:44:20] was 36 to 50 megawatts. [00:44:24] Today, you hear about this gigawatt, that gigawatt, this 2.2, that 11. [00:44:29] If, when you introduce this type of these modules where you can drop them into legacy facilities, [00:44:36] when you think out in three years, what do you think the average size of a data center build [00:44:42] will be? [00:44:43] Will it be a hundred megawatts? [00:44:44] Will it be a gigawatt? [00:44:45] Or will it be back down when you layer in all the inference and all these modules? [00:44:49] And you think about that. [00:44:51] What is the average size build? [00:44:54] Because I think we get distracted with lots and lots of these crazy numbers. [00:45:00] We're showing what our chip, our system technology is capable of. [00:45:06] But as we create all that data, nobody wants to throw it away. [00:45:11] So next to our stuff will be storage. [00:45:15] And just because they're robots while you're sleeping, talking to each other, [00:45:19] they still need to log on to the other data centers around the world. [00:45:23] So my prediction will be that you're going to have, you know, it's easy to put this on the edge. [00:45:29] It's easy to put the high power where you need it. [00:45:31] You're going to have high power zones that are larger and larger models. [00:45:35] But I think what I see and why we're donating this is I think we need this late binding mixed environment. [00:45:41] So that way, let's say that shield is 10 megawatts. [00:45:47] You don't want to have to change your data center. [00:45:50] You want to be able to go to the perimeter of the data center where you had your eight to make 600 amp feeders at 1600 amps. [00:45:59] And you say, okay, that has aged. [00:46:02] I got a good life out of it. [00:46:04] For 60 cents a watt, I build a new infrastructure that shims into the high density. [00:46:10] So my point is I'm trying to drive it so it doesn't matter. [00:46:14] So that is that if we can build these modules together and someday there will be solid state transformers for everybody. [00:46:21] You get a solid state transformer, you get a solid state transformer, and it'll be 800 volts. [00:46:25] Because that's the right answer. [00:46:26] So you're going to go straight from the grid at 35,000 volts. [00:46:29] We're going to talk about that at OCP. [00:46:30] Do I keep, Rob, did I get enough plugs for you? [00:46:32] Straight from the grid, our IT will be 800 volts. [00:46:36] And it'll have all the energy storage in it. [00:46:39] But you're still going to have IT that's older and that needs it. [00:46:42] So we're trying to design the perimeter once, the 10 megawatt perimeter, I think 6.2, I said there, or the 20 megawatt. [00:46:51] And as you build out a giant campus and say, okay, this client came in and he's a large language model builder. [00:46:59] He wants it, he or she wants it all, you know, as one system. [00:47:02] And somebody else is saying, oh, I'm a diverse cloud. [00:47:05] You know, I want a third of that for training models. [00:47:08] I want a third of that for inference for two models. [00:47:12] I want, you know, 20% of that for lots of enterprise inference. [00:47:17] And I need a gigawatt of storage to go with it. [00:47:21] Before we get to our last question up here on the second floor, I do want to call out that we have the QR code up there. [00:47:28] If you have a question that has not been gotten to, you can submit that online. [00:47:31] We will get to your question by podcast, by social media. [00:47:34] We'll see those questions. [00:47:36] So if you haven't had a chance to ask your question, scan the QR code and submit it. [00:47:39] Now up here to the second floor. [00:47:41] What was your question? [00:47:43] Hey, somebody got a mic up there. [00:47:50] What will modular data systems look like in five years? [00:47:53] Was that correct? [00:47:54] Data center. [00:47:55] Sorry. [00:47:56] Yes. [00:47:57] Well, the answer is there's so many data centers being built at any one time. [00:48:02] It's, you know, they're all, they're all morphing and we've got existing data centers. [00:48:06] So, so we think with this technology that we're describing and allowing all you folks to participate in it, we're showing that, you know, we can support the highest density in an existing data center. [00:48:19] Now, if I knew if every rack was a megawatt of rack, how different would the data center be? [00:48:26] A generator's got to breathe. [00:48:27] A generator's got products of combustion. [00:48:30] Every volt amp in has to go out as heated air. [00:48:33] So, so all of that stuff around the perimeter that is at 30% waste, that has a certain amount of physics associated with it. [00:48:41] So we might have a lot of pickleball courts space inside when all of our IT collapses down. [00:48:48] But I don't know. [00:48:49] I mean, even if you're starting from scratch and building it with robots, maybe if you're, and we're 800 volts DC straight from the grid, native DC, it could be different. [00:48:58] Maybe if you can take that waste heat and, and, and, and put it into a, you know, a river or into the ground so you don't have heat rejection. [00:49:07] So, I mean, I think those answerly things are going to change data centers more than the IT. [00:49:12] Our, the IT that we're building is going to continue to make tokens at a profitable rate for everybody all the time at any scale. [00:49:20] So your mileage may vary. [00:49:22] Greg, Greg said, yes, about modular. [00:49:27] Oh, oh, Greg. [00:49:28] Oh yeah. [00:49:29] So, so I think that was Leslie up there. [00:49:30] I said, no modular. [00:49:31] This is modular. [00:49:32] You, you, you just, you, you put it inside a stick building. [00:49:35] You wrap it around your own bespoke container. [00:49:37] You know, that, that, what we're given here, what we're showing here, this is how you, this is how you feed, right? [00:49:44] This is, you know, again, the, the yellow wire matters and the blue wire matters and the power wires matter in the water. [00:49:49] It doesn't matter how you deliver it. [00:49:51] You can deliver it in Greg's module. [00:49:53] You can deliver it in Jim Simonelli's module. [00:49:55] You can deliver it in, in, in Teleflex, compute, compute dynamics modules, or you can deliver it in, in Wade in a t-shirt, right? [00:50:03] It doesn't really matter. [00:50:04] As long as I can plug in somewhere, you know, I'm good to go. [00:50:11] Was that it, Leslie? [00:50:12] Did I get you, get your answer? [00:50:14] Okay. [00:50:17] Well, thank you so much. [00:50:18] I appreciate all the time and let me come up here and share with you.