Delta 800 VDC Power and Cooling Solutions for Data Centers presented by Delta

[00:00:00] Ralf Pieper: Okay, yeah. Welcome to OCP and to our Delta Executive Session. My name is Ralf Pieper. I'm the R&D Director of CDBU, Custom Design Business Unit, inside the Delta Power and Systems Business Group. And today I have the honor to have two experts on stage. So, Harry from... [00:00:29] Harry: Yeah, from Accelerated Computing Group. And Xichung. [00:00:34] Xichung: Yeah, Xichung from Delta. Can you hear me? I don't know. From Delta, I think, my microphone is not on. [00:00:43] Ralf Pieper: Yeah, you don't have this microphone. This is just the Delta one. [00:00:47] Speaker ?: All right. [00:00:48] Ralf Pieper: Yeah, you need two. [00:00:49] Speaker ?: I need two. [00:00:50] Ralf Pieper: Okay, so you have time enough, right? I, I, I, I... Okay, yeah. So, together we want to present to you how we empower the next generation AI computing with the Delta 800 volt DC power and cooling solutions for data center. So, the next 25 minutes, yeah, we will start with Harry's point of the AI, how it shapes our future with the view from Nvidia. And then we will show the grid to chip power architecture for the current 50 volt DC installations. And, um, to empower gigawatt AI data centers, we will introduce the 800 volt DC infrastructure. And, um, at the end, Xichung will, um, show us the cooling solutions from Delta, um, yeah, to help powering this gigawatt data centers and cool them. And, um, so I will hand over to Harry. Great. [00:02:02] Harry: Okay. [00:02:03] Speaker 4: Thank you, Ralph. All right. So, I'm excited to talk about this, uh, really kind of once in a lifetime opportunity that's presenting us. I'm going to explain the context at the stage, uh, for why these power and cooling solutions from Delta are going to be so important. So, on this slide, you see that we've reached the turning point in terms of data center investment, um, the folks at Del Oro, uh, they project that total spending will reach $1 trillion, uh, in just a few years. Uh, that inflection point is driven by the exponential growth demand for AI infrastructure and accelerated computing is at the heart of this growth. So, the scale of this transformation is changing the economics of data centers, and also the strategies of every enterprise that's building AI capabilities. So, let's just take a quick look at how we got here. Um, in the beginning here, in the early days, AI models were limited by human labeled data sets that constrained the amount of data. And therefore, the performance also of the models, um, but with the advent of transformers embedded, uh, unlabeled data, uh, became usable in model capabilities, uh, grew dramatically. Uh, this unlocked the generative AI boom, and, uh, that's where the models could take multiple modalities, texts, video, um, uh, images, and then generate outputs in those various formats. Um, today, the leading edge is in mixture of experts models. And so, these are where multiple specialized models, um, collaborate to produce the best answer. Um, and deploying MOEs into inference brings enormous infrastructure demands. Real-time response, trillions of parameters, and, uh, massive input sequences. Now, all of this will test the limit of data centers. Our modern journey with AI began with AlexNet in 2012. And so, in the next, like, 13 years, um, since the big bang of deep learning, um, we've seen three distinct waves. So, there's perception AI, generative AI, and now reasoning. And perception AI gave us speech recognition and medical imaging. Uh, generative AI brought us the ability to create, uh, these various multimodal formats. Intelligence, however, requires reasoning and planning. And that's where agentic AI, uh, comes in. Systems that not only generate, but apply logic, make plans, and take action. And more and more, taking action means doing it in the, in the physical world. And, uh, and so that, that physical manifestation is in, like, autonomous vehicles or robotics. Um, to support this leap, on the other side of the chart, you see that the researchers discovered three scaling laws. So, we talk about, um, pre-training, post-training, and test-time scaling, or, uh, long-thinking. And each one describes how you apply more compute, and you get higher capability. Um, but meeting these demands means orders of magnitude, more computation at data center scale. So, you see a hundred X on the, on the chart. Fueling the need for accelerated computing and AI factories. And that's kind of what we're going to be talking about building. Let's tie this to economics, though. So, on the left, imagine you've got, uh, GPU, a quarter of the performance of the Blackwell, uh, GPU, and it's free. Um, a workload like DeepSeq R1, you could realize $8 million in revenue over three years, uh, with that. But if you were investing $3 million in a GB-200 NVL-72 platform, you could instead generate $30 million over that same period. So, the lesson is, um, free compute is not nearly enough. Um, performance is the lever that, uh, decides what AI inference economics is. And that means, you know, throughput in the infrastructure translates directly into higher, higher ROI. And all of this is sort of explaining why there is this big investment boom and why, um, the technology at scale matters. So, if we look ahead, NVIDIA started talking about, uh, a one year architecture cadence for technology. We're aligning our release, uh, release cycles to the really crazy pace of AI innovation. Um, so, at the foundation is the compute silicon. Um, you've got Ruben, Ruben Ultra. We just announced, uh, Ruben CPX for context-rich workloads. Uh, the connectivity, um, also has to advance just as rapidly. So, NVLink 5, 6, uh, Spectrum X, um, uh, uh, ConnectX, or CX 7, 8, um, these are for scaling out, uh, to hundreds of thousands of GPUs in a, in a data center. And, uh, all of this performance, though, uh, comes at extreme density and power demands. So, systems like NVL 72, NVL 144, NVL 576, uh, they'll be used to concentrate hundreds and thousands of GPUs into a single AI factory. So, that's why innovations in power distribution and advanced cooling, like with our partners here, are really critical, as critical as the GPUs themselves. And, uh, that's what makes us able to deploy accelerated computing at scale. Um, so, with that, I'm going to hand it over to Rolf, uh, who will talk about the power and cooling solutions. Yeah. [00:08:09] Speaker ?: Thanks. Thanks. [00:08:09] Ralf Pieper: Thanks, Harry. So, Delta at a glance, um, in the last, or over the last five decades, uh, we came the world's leading provider for power and cooling solutions. And, uh, for these two topics, we will concentrate today in, in our presentation. So, as I mentioned before, I will start with the current deployment with the 50 volt voltage for the servers. So, here on this page, you see the, um, single line diagram, um, from the medium voltage down to the, uh, core voltage of 0.65 volt. So, from left to right is the power flow. Um, the power shelves are deployed in the IT rack with this 50 volt solution with the 50 volt bus bar. Um, and the, uh, power supplies are normally powered by, um, three phase AC, but we have six power supplies and each one is running between line and neutral. So, we have single phase power supplies, um, generating AC to this 50 volt. Um, we can also deploy, this is the, um, PCS you see below the rack. We can deploy the cap shelf, um, to cope with the, um, very high dynamic, um, load jumps, uh, during the, uh, processing and, um, yeah, calculating of the GPU. So, this helps, um, to smoothen the AC grid. Yeah. And makes the, uh, energy provider happy. So, they don't like to see this, um, jumping on the grid. Um, and then on, uh, server level, we have the 50 volt to 12 and 12 volt to 0.6 volt, um, yeah, downstream DC DC converters. Um, you can imagine if the, uh, IT rack power goes up to, uh, one megawatt. 50 volt is not the right choice anymore. So, um, the, um, currents would be 20,000 amps, uh, which is, uh, way too high. So, to empower the future, um, we developed together with NVIDIA and Meta and Microsoft and Google, um, the 800 volt DC, um, yeah, power infrastructure. Um, here you see on the left side a typical AI rack. And, um, of course, um, this is the AI rack. So, it's full of AI servers. So, three quarters, um, is occupied by the payload. And there is no room for additional power shelves, or room for BBUs for battery backup, or the super cap shelf for smoothening the EDPP. So, the solution is to move all the power-related stuff out of the IT rack into, uh, we call it in-row power rack, or sidecar. So, here you see the, uh, transition plan from the, um, 800 volt DC. So, it goes from top to bottom. So, in the top line, you see the current deployment as set with the 50 volt. And if we are, um, higher than 250 kilowatt for the IT rack, um, we recommend, um, to move into this in-row power rack. And, uh, use 800 volt DC output, which is provided, um, to the IT rack by a crosslink cable. So, we will see this on the next slides. And, um, the, um, the green box in the second line in the IT rack. This is the downstream, uh, DC/DC converter from 800 to 50 volt. Because, currently, uh, only 50 volt native servers are available. Um, but, of course, um, all the companies are also working on a 800 volt native server. With the 800 to 50 or to 12 volt, um, on the tray level. And, uh, if this exists in one or two years, I guess, then we also enable the, the bottom line. So, it's, um, solid state transformer, uh, directly converting the medium voltage into this 800 volt DC. Which is distributed then inside the data center. So, instead of the AC distribution today. So, this is our vision. And then you can hook up directly the IT rack to this 800 volt DC. So, on the next slides, I will show the, um, the parts which are involved in this, um, in-row power rack. So, it starts with the AC entry box or AC PDU. So, there will be, uh, three, um, AC drops. Each one is, uh, 200 amps. This is coming from the busway, from the ceiling. And, uh, this AC PDU splits this, um, 200 amps into four times 50 amps. With some, um, upstream protection. Um, this outputs, the 50 amps outputs, um, go directly into our power shelves. So, we have two versions. So, the bottom one is a 21 inch, um, with six power supplies. And the top one is the 19 inch, um, with four power supplies. So, the top one is 106 kilowatt. The bottom one is 180 kilowatt. So, the AC comes in. And then, um, the power supplies convert the three-phase AC. So, it's a real three-phase without neutral. We convert to 800 volt or a floating 400 volt. Which we can combine two units to generate the plus minus 400. So, this is, um, Diablo 400 compatible. And, um, all the output of the power shelves and the BBUs and the, um, CBUs are collected together with this 800 volt DC bus bar. So, you see the red, the dotted blue, and the black conductor. So, it has three conductors. So, this brings the power up to this DC PDU. Inside the DC PDU, there are also fuses and relays to remotely turn on and off the four outputs. So, you see this blue connectors. And, um, you see on the bottom left, uh, this is the crosslink cable. So, on each PDU, we can connect four crosslink cables, which go then to the IT rack, where we have the 800 volt to 50 volt DC DC converter. So, it looks similar like the HPR shelf, but it has three times more power. So, this is a 108 kilowatt in one U. And, it has two times this AC, sorry, the high voltage DC inputs. And, the output is the bus bar connection in the middle 50 volt. Then, for the long term, as I said, um, the solid state transformer, uh, will convert directly, um, the medium voltage to 800 volt. And, the beauty of this 800 volt DC distribution is that we can hook up easily renewable energies like fuel cell or solar. And, we can also deploy, um, central battery for, yeah, like, acting like a UPS or also for, um, smoothing the grid. Um, then, as I also mentioned, the 800 to 50 or 800 to 12 volt, uh, will be on tray level. Here, you see a 12 kilowatt, a PDB. And, then, the, the last step is, um, again, a P, point of load converter, um, next to the GPU ship. Okay. Then, I hand over to Chi Chung. Thank you, Ralph. [00:16:14] Xichung: This one, right? Mm-hmm. All right. So, good afternoon to our guest in the room. Uh, let me introduce myself again. My name is Chi Chung. I'm a sales product manager from Delta Fang and Thermal Management Business Group. And, I've been with Delta for nine years and had an honor to participate in the thermal design transition with NVIDIA. From what used to be just a fan and heat sink on a product of a graphical size to now a complete liquid cooling system in a, in a rack for a rack in a data center. So, well, my colleagues from power supply team, they are working around the clock to bring out the best power solutions. Let's not forget importance of cooling too. So, next in my section, I will introduce to you Delta liquid cooling solutions. I will talk about our corporate design focus for the broad level solution and also our CDU for rack level solution. With only limited time today. So, if you find any of the material interesting to you, you are more than welcome to visit our booth at number B51. So, first of this page, it gives a look over the solutions that we can provide in the data center. So, broadly speaking, it can be divided into three scope, broad level scope, rack level scope, and system level scope. So, in the broad level scope, in the broad level scope, we focus on the thermal solutions for the hotspots inside traces. Usually, the CPU, GPU, NPU, ASIC, or the peripheral hotspots like optic transceiver, memories, SSD, sorry, SSD, and so forth. Which we can incorporate either with Delta in-house developed DC pump to make it a closed loop solution, or with different type of host routing like PTFE, EPDM, stainless steel, and different type of quick disconnect to design an open loop solution. Depends on system requirement. In the rack level, there are three major offers we are providing to our customer. The in-rack liquid to air CDU, in-rack liquid to liquid CDU. And with the power thermotypes have power rack density. It's going up to a future where there might be no more room to be spared for in-rack application. We are now bringing out our DC driven in-road CDU. So with the readiness of the board level solution and the rack level solution, together with the design of rack manifold, basically we at LCSBU can provide thermal solution for entire rack as a whole. So when it comes to cold plate design, there are many different approaches we can adopt to adjust the heat from the underlying hot spot. So for different level of power density, there are different cold plate focus. So in scenarios, in cases where the power density is less than 20 watt per square centimeter, we can simply just bury a water loop into either a aluminum plate or a copper plate to adjust the heat. But when the power goes up to around 7 watt per square centimeter, then we can further incorporate the thin block with some straightforward flow channel, we call it cross flow channel design, into a cold plate to increase the liquid contact surface to meet the required thermal performance. And when the power is go up, and when the power is go up a notch to around 100 watt per square centimeter, with the thermal design power per square around 2.8 K watt. Then the most sophisticated flow channel design will be considered. For example, impinging flow, in this case, instead of when the liquid flow into a cold plate, instead of just letting it pass through the thin gap from head to toe, we can direct a liquid, use a partition to direct liquid to the hot spot first and use the heat to the hot spot when the coolant is in the lowest temperature state to enhance the coolant efficiency. Or we can use partition to separate the side channel and flow channel to extend the flow distribution travel path in the cold plate to provide a more widespread flow distribution deployment in exchange for a better performance. And today, we are seeing the thermal design power is going higher, like 100 watt per square centimeter, with the ASIC TDP nearly 3.6 kilowatt. In that case, right now, our team, we are looking into the possibility of increasing the thin pitch up to 0.1 millimeter to maximize the thin contact surface in order to hit a lower thermal resistance. And also, if the total keep-out zone allowed, we can also explore the possibility of incorporating vapor chamber into a cold plate to take advantage of the vapor chamber's phase-changing characteristics to enhance the heat spreading efficiency in both x, y and z direction to achieve the lower thermal resistance. These are the cold plate solutions we are preparing. And aside from cold plate, aside from cold plate, CDU also plays an equally important part in the application. So right now, this page gives a look into our current or upcoming in RECCDU to offer. All are DC driven. The top row one, the four units are in liquid to liquid application. Right now, we have MPed the 60 kilowatt, 60 kilowatt for you in RECCDU and 140 kilowatt in RECCDU. And right now, we are going to launch our next generation 250 kilowatt in RECCDU. And the two units at the bottom, not the liquid to air. And what we are going to launch is a 24 kilowatt in REC liquid to airCDU. As I mentioned, right now, we are seeing that because of the AI, the power rack density is trending toward the future where there might be no more room that can be spared to install in RECCDU. We are going to launch our in RECCDU. So by the end of the Q1 next year, Delta is going to launch our stake up in RECCDU. In this design, we are going to stake up six units of our 250 kilowatt in RECCDU in one rig with a centralized control to serve for the same purpose as in RECCDU. But the benefit is our customers don't need to invest in a whole in RECCDU at once. And the target cooling capacity we are pitching is 1.5 kilowatt with this unit. And in Q3 next year, we are going to launch our own in RECCDU with the DC Delta in-house designed DC pump that can directly run 800 voltage DC. And the pitch, the cooling capacity we are pitching to adjust is 2.4 kilowatt. Megawatt. [00:23:14] Ralf Pieper: Sorry. [00:23:15] Xichung: I'm sorry. Yeah. Good. Thanks. So these are the cooling information I would like to share. Thank you. And I will hand back to Ralph. [00:23:26] Ralf Pieper: Yeah. Thank you, Chichung. All right. Yeah. This brings us to the end. Thank you very much for listening. Our booth is B51. And we are showcasing this 800 watt DC in-row power rack. So please feel free to visit us. And yeah, of course, we can talk and answer your questions. Thank you.