Webinar: Data Centre Liquid Cooling Technology

[00:00:00] Speaker 1: All right, let's get started. Hello, everyone. I'm Larry DeAngelis, VP of Marketing here at Park Place Technologies, and thanks for joining us today for our webinar, How Liquid Cooling Can Help Organizations Deliver Sustainable Data Centers. Two great speakers today, Chris Carrero, who's Chief Technical Officer at Park Place, and Chris has been driving our entry into the liquid cooling space for the last year or two. We also have Manfred Chua, VP of Business Development at Zootacore. Zootacore is our partner in the direct-to-chip cooling space. We have a lot to present today, but certainly encourage your questions, and we've got a few in advance, so thank you for those. If you have questions that you haven't submitted yet or they occur to you during the presentation, your mics are muted to avoid and minimize distractions, so please use the team's chat function, and sometimes we'll address them if they hit at the right time in the flow of the presentation. Otherwise, we've got time at the end to address questions submitted previously and during the presentation. So, again, welcome, and with that, I'll hand it over to Chris Carrero. [00:01:19] Chris Carrero: Awesome. Thanks, Larry. Welcome, everybody. I appreciate the time today to go over some of these topics in becoming sustainable in the data center. So, it's pretty interesting. We're at an inflection point where not only CPUs, but GPUs as well, can't be cooled or coming up to the mark where they can't be cooled with air anymore. This graph actually shows how we're at 250 into 300 watts per CPU air starts dropping off and requiring actually liquid cooling to remove the heat. It's actually crossed over a little while. There's been a lot of innovation in the fan within the actual server to get enough CFM through the case, but we're getting close to a point where that's not going to work anymore. And that's really been one of the big drivers to pushing towards liquid cooling in the data center. Another thing here, another great graph that kind of shows how we've been pretty stagnant with PUE. PUE is power usage effectiveness. It's basically a reading. The way I like to look at it is if you were to check the meter outside of the data center to see how much power is being drawn in to run it. So that includes air conditioning, lights, the actual, the full, all of the cooling equipment in the servers inside of the data center. So switches, servers, and all the air conditioning components. And as you can see, since around like 2014, there's been some mass improvement from 2007 to 2014, but we've been pretty stagnant. 1.65, 1.58 PUE, and we've gotten slightly more efficient, but here at, on this chart up to 2022, we're right around 1.5 is the average annual power usage effectiveness for data centers globally. And with, with that, a lot of this, it's actually 40, 40 plus percent of power is consumed just in the air conditioning alone in the data center. So it's a significant amount of power is, is used just for air conditioning. [00:03:45] Manfred Chua: Yeah, and if I may add, Chris, a few things that, that are, are happening, and I know that the, the chart here ends at 2022, but, you know, things that have been happening over the past, past few years, the scale of data centers has, has, has been, has been increasing dramatically, especially with the, the emergence of, of gen AI and the AI factories. So you're going from, uh, what typically was 10 to 15 megawatt size data centers to, uh, you know, to, to, you know, 50 to 100 megawatt data centers. So the data centers are getting bigger. And of course, with that, uh, you also have regulations around the world, uh, from a sustainability perspective that are in certain jurisdictions, mandating, uh, PUE for new data centers. And so when you take a look at, uh, the fact that the data centers are getting bigger, but also you've got PUE regulations, uh, that is creating a lot of tension, uh, around the fact that, uh, data centers are needing to grow, uh, especially, you know, given the fact that it's, these are, you know, the lifeblood of certain companies. And, and, and, and, and so, uh, it's putting a lot of emphasis around, uh, what are the technologies like liquid cooling, for example, uh, that can enable that growth while fitting in, uh, these PUE envelopes. [00:05:14] Chris Carrero: Right servers, uh, traditionally, so, you know, in 2000, uh, or 2020, 2019, you're looking at 2000 watt servers, maybe three or 4,000 watt servers at the highest. Um, and now we're talking 15, 14, 15,000 watt servers. Um, and then you're putting those in racks of five or six, you know, it's, it's really creating, um, you're right. It's really driving the demand, um, in, in AI is driving it in, in it's, all it's doing is forcing the requirement for the need for more sustainable solutions. So one, one thing to compare, so we're, to get into kind of the liquid cooling, now that we know some of the reasons and drivers, um, Park Places has worked and have been working for the past two years now on, um, different solutions. Um, just to kind of talk a little bit about them, we have, um, on the left here, you have an immersion cooling solution. So this is actually where you take a tank of a dielectric fluid, you have, um, heat exchangers, pumps, um, in, on the inside of the building. And then you'd have an evaporative cooling tower, um, or a chiller or something else within a water loop outside, um, outside of the building. So you take the server, you directly immerse it inside of the dielectric. It, the heat would be, uh, removed with using that dielectric as it's kind of flowing around the server. And it, it's about a thousand times better at pulling heat than, uh, than air, than, than putting air around the server chassis. So that heat exchanger is basically, um, either built into the, to the tank itself, um, or it would be in a row of tanks. So you could have multiple back-to-back four or five tanks connected to that, um, it's essentially, it's called a CDU, but it's the basis inside of it. It's a heat exchanger, um, with some pumps. So, and then to compare that to a direct-to-chip, which is another alternative, um, with direct-to-chip, you have a CDU, which is basically pumps, some, uh, some type of liquid. And, um, that circulates outside to a chiller or, um, an evaporative cooling tower. But, um, the difference here between the two is you have, you have cold plates on the inside of a server. So a cold plate is, is nothing more than a, a block of, um, for a single phase, it's a block of metal, um, that takes the heat from the CPU or GPU, circulates back to the seat, to that CDU, that coolant distribution unit, and then it, um, passes off that, uh, heat energy out to the chiller or evaporative cooling tower. So these, uh, these are two of the, of the bigger solutions, um, as it comes to, um, to cooling or more efficient solutions, uh, that are available. Uh, there's others such as rear door, um, and slightly more efficient, um, uh, systems. But these are, these are a lot more efficient because they're pulling heat directly from the system or the components that are, uh, that are producing the heat, not indirectly like, um, an assisted air cooler or, or something else. So, um, this slide's actually a pretty powerful slide. It shows the, kind of, the significance of what an immersion cooling data center would look like. Um, a couple things to notice here. So you don't have raised floor, you don't have crack units, less generators, um, really this comparison here, you're looking at like a legacy air data center using, um, 17, 1700 kilowatts of power to generate a thousand kilowatts of compute. So that's, that's, that's quite a bit of infrastructure. So there's a lot of costs associated to that where in a green field, if you're to build a new data center, that's, um, strictly immersion, um, you only need, uh, 10, 30. So 1,030 kilowatts of power to generate that same requirement of a thousand kilowatts of compute. Um, but there's a lot of cost savings with less generators, less, um, less racking, um, less infrastructure. You just have five tanks roughly on the floor to, to actually produce that same amount of compute. [00:09:41] Manfred Chua: So, uh, so, uh, so why, why do we need liquid cooling? Um, and I think that the previous slide was a great example, right? We're going from, I think the previous slide had an example of a, a PUE, uh, roughly like 1.7 getting down to sub 1.1, right? So getting down to a, a less than one, less than 10% of your, your power, uh, going towards, uh, cooling. And, and this is a, a, a, a great chart that shows the various technologies being compared to air cooling. But before I get into that chart, I think that, um, it'd be great to just spend a little bit of time just, you know, talking about how we got here. Right. And, and, and part of it is, you know, liquid cooling was formerly in the domain of, of high performance computing. Um, you know, so it was for a long time, uh, an HPC thing, right? Uh, and, and, and HPC has been using, uh, liquid cooling and specifically direct to chip liquid cooling, uh, for a long time. And, uh, when, when you take a look at, uh, the, the needs of, of the industry, when generative AI came out at the end of 2022, uh, the, the, the fact that there is such a demand for generative AI and also the, the large language models and the AI models that, uh, are, are being set up. Now, those models are growing in size at such an exponential rate that when, as those models are, are, are growing, it's driving the size of, of data centers. Uh, uh, uh, uh, and as those, the size of data centers are growing, um, as, as I said before, we're going from, let's say, five, you know, five, 10, 15 megawatt data centers to these AI factories. Uh, and when you move from that, that type of scale, uh, you still need to start looking at optimizing around land, around power, and around water. Uh, and when you start looking at optimizing around that liquid cooling becomes an essential, uh, uh, an essential technology in order to, you know, make all of this work, uh, from an ROI perspective. And so when you, when you take a look across the different, uh, the different technologies, right, direct-to-chip single phase, uh, direct-to-chip, uh, dual phase, which is what pseudocore, uh, my company, uh, specializes in. And, of course, uh, immersion cooling, where you take the entire server board and you immerse it in a dielectric fluid. Uh, there are pros and cons across, uh, the three. Uh, I'll spend most of my time on direct-to-chip, uh, however, from an immersion cooling perspective, I will say, uh, that, uh, there, there is, uh, there is a, I guess, time and place for everything. Uh, although the, the main thing for immersion is, uh, that, you know, there's, there's a lot of material compatibility testing that needs to occur. Because, again, all of the materials that are on that server board, uh, you know, need to be vetted in order to make sure. Yeah, from a compatibility standpoint, uh, you know, before I joined Zootocore, by the way, I was, I was at Intel and, and I drove a lot of the programs for our, our liquid cooling, uh, and, and, and specifically immersion, actually. And so, uh, you know, I think that there is a lot of great applications for immersion. [00:13:37] Chris Carrero: But getting back to- It worked well in a lot of the edge, uh, deployments. So, I mean, that's, that's a really, uh, good place where we're starting to see some activity, um, and where, uh, immersion's starting to shine. Um, it's been around for quite a while. It's taken a while to kind of get to this point. But now with some of these edge deployments where you don't need an air conditioner, um, and, and then you could actually, like a closet edge, like very small, less than 10 cabinet type of deployment where instead of spending the money on air conditioning, um, and, you know, running out of space, you could actually, um, reduce the, you could, you could actually run a, an immersion tank that protects. The server from the elements and it, and it sits in a closet. So yeah, definitely. You're definitely right. It's a certain place, um, for each of the solutions. Exactly. And I think that where, where, of course, uh, and, and I would say where, where direct to chip. Is really finding its ground now is in the space of, you know, the, the domain of AI servers, right? [00:14:40] Manfred Chua: Because as you see, as you see your thermal design power, your TDP of your AI accelerators, um, go higher and higher. So like from a heat density, heat flux perspective, uh, the direct to chip technology is able to actually evacuate that heat very efficiently. And, and again, you know, if you take a look at the history of direct to chip technology, it's really been focused on that type of application, right? Taking a very focused amount of heat and evacuating that heat. And so. When you look at the AI accelerators today, their thermal design power, uh, where processors CPUs were in the 300 or 400 watt TDP range. Your thermal design power of your accelerators are easily double or even triple that or will be soon. And so that is, is, uh, lending itself very well to the DTC, uh, technology. And so when you have, uh, and so this, this just shows, uh, our version of the CDU. Um, we call it an HRU, a heat rejection unit because we use a two phase technology where this is really our condenser unit. And so, uh, we have an air version where if you don't have a primary, uh, water loop, uh, available where your server rack is, we actually can just, um, uh, have a, an air version that, that can support it. But if you do have a primary water loop that we can hook up to, we have a higher capacity. And so we have a, uh, HRU water version that has, of course, a higher capacity cooling capability. Uh, and then of course we have a rear door, uh, condenser that, um, that is of course on the right. Uh, and then, you know, from, uh, from the benefits of, uh, DTC cooling, uh, overall, I mean, we're able to really, you know, drive the, the energy efficiency and really get down as, as, as Chris was showing in the previous slides, uh, from a stagnation of 1.5. Uh, you know, Zootocore actually has, has seen and, and, and done, uh, some, uh, uh, uh, white papers within OCP that we've gotten down to 1.05, 1.06. So that's really 6% of the energy from a PE perspective, um, that would be dedicated to, uh, you know, to cooling and that's in a modular data center. Uh, and, and that type of energy efficiency is really lending itself to, uh, being able to optimize. And if you start scaling that out at a hundred megawatt type scale, that is significant in terms of the, the energy efficiency you're going to be able to grab. And then, uh, when you look at from a, a water perspective, uh, you know, there, there was a comment in one of the other slides around hydrophobia. So hydrophobia, uh, really manifests in several ways, right? So there is in general, uh, hydrophobia is really around the fact that you have water in the data center. And so water and electronics don't mix very well. And so when you have water and, and electronics, you have not only, of course, the, the, the risk of water and IT equipment from a leak perspective, but you also have, uh, the, the risk of corrosion if there ever was a leak. And also you have, uh, the maintenance involved in keeping that, uh, that installation going from a, from a, uh, uh, installation perspective where you have bio growth. You've got, uh, all these other things that you have to maintain. And so from, uh, a two phase perspective, which is, uh, where Zootacore is, we have a, we use a dielectric fluid. So it's a non-conductive dielectric fluid. So if there ever was a leak, it basically goes onto the equipment and evaporates into the air. And so there is, uh, no risk to the IT equipment. There is no corrosion factor. There's no bio growth factor. Uh, and we significantly eliminate, uh, the, the, the hydrophobia, uh, from, you know, from the equation. And, you know, our technology of course is, you know, two phase. So we use the phase change of the fluid to evacuate the heat. So from, uh, the fluid to vapor change is what is evacuating the heat versus a flow. You know, the radiant heat, uh, mechanism, which is what single phase is. So this is describing, you know, the two types of directed chip cooling. So you've got single phase, which is basically you're, you're relying on flow to, so you've got typically either water or a glycol mixture of water, um, that basically flows across the plane of the chip. Um, you know, and then, and, and you're relying on that flow to evacuate that heat. And, and so as the chip gets hotter, you turn up the flow so that you're evacuating more heat. Um, whereas, uh, our two phase, you're relying on the boiling of the water or not, sorry, not water, the boiling of our fluid. So that the, as the, the chip gets hotter, the boiling just becomes more rapid. And so again, two different approaches, one, uh, relying on a more water based approach with flow and the other relying on a dielectric fluid that has. Uh, uh, uh, you know, a boiling two phase, uh, approach. [00:21:01] Speaker ?: Yeah. [00:21:01] Chris Carrero: So talk a little bit about, um, park places solution in, in some of our partnerships. As I mentioned, we partnered with GRC. We've partnered with the OEMs. We've partnered with Zootacore. Um, we're basically driving, um, a full turnkey solution for, regardless of, of what the need is for liquid cooling. Um, we take an unbiased approach. It's basically, you come to park place. We can help with procurement, conversion, installation, um, make the maintenance portion. Um, a big piece of some of these really innovative technologies is you lose your OEM warranty. And, um, the OEMs willing to work with us to actually get us the system, give us the firmware that we need to, to get on these systems and still offer that to the customer. Um, but we, we would pick up the warranty while they're trying to figure out this. Um, I would call it like this gray area in between now and when they eventually adopt it. Um, we've seen in, uh, in a couple of, um, different situations on the immersion side where the OEM is actually offering a warranty. Um, and I, I figure it's just a matter of time before everybody has, um, some type of adoption for some of these advanced technologies. Um, but, um, park places approach, or at least taking, as I said, the unbiased approach, we can look at a customer's, um, current situation. So they're either, it's a greenfield data center. Um, it's an edge deployment. Um, it could be a multi megawatt, 28, 30 megawatt build. Um, we could help with guiding them in the right direction, um, giving them TCO calculations. So looking at them from different angles, from different total cost of operation. Um, some of these solutions are, are pretty expensive on the front end, but if it's a greenfield deployment, you need a lot less cooling. Um, so you can actually save money, um, from the start with going with one of these innovative technologies first. Some of the retrofits, um, require, um, conversion, uh, in a, in a depot. You got to remove the, the thermal compound. Um, you have to either fix a cold plate to it. Um, some of the tolerances are, um, are, are pretty critical. The, some of the bigger systems are HGX platforms. Uh, as Manfred mentioned before these, um, these AI factories, these large deployments, um, they're running eight GPUs. Like an example would be like a Dell 9680. Um, you have a, um, actually I'll flip slide here just so you guys can see here. A 9680 is a, um, it's this model is an HGX version. There's eight GPUs on there. Um, you would need to affix these cold plates. They're pretty close tolerances. Um, a lot of this, this work needs to get done. So Park Place offers the solution. We partnered with Zootacore, um, for actually doing these, um, these builds on the front end. And then also the maintenance, um, moving, moving forward. We have trained engineers to go out and, and service these types of systems. The OEM, uh, depending on the OEMs, HP has a version of this. Super Micro has a version of this. We've partnered with all the main tier ones and, um, all the, uh, the major tier ones on some of these deployments. And, um, basically to sustain whatever type of, uh, the customer, um, desires or what kind of systems they want. [00:24:37] Manfred Chua: Yeah. And, and thanks for, uh, putting this slide up, Chris. Uh, so as, as Chris mentioned, uh, so, uh, the, the, the picture on the, on the top, top right is actually our, our, uh, our, our cold plate for the GB 200. And so that guy, um, what's, what's interesting is it's massive. So if you guys have seen the pictures that Jensen always, uh, you know, uh, shows in his, uh, in his keynotes. So this, this cold plate is actually like this big, right? It's a, it's a monolithic cold plate that, that actually covers the two black wells, the, the, the grace, uh, you know, the grace, uh, CPU. And then, uh, some of the IO, the IO parts, but basically it's 2800 Watts of, of cooling, um, in, in that, in that, uh, cold plate. And all of that is basically, I mean, think of it as, you know, a, a copper plate on top of all that silicon and then a pool of dielectric fluid, uh, that basically is boiling across that plane to evacuate the heat of that 2800 Watts. And, uh, it's, it's, it's, it's pretty interesting to just, uh, see such a big, of cooling. Um, now that same principle, of course, applied to whether it's the H two hundreds, which are the ones that you see directly below or even processors, which are, which is the cold plate. I believe that you see on the, on the left. So. Yep. [00:26:13] Chris Carrero: Those are the CPUs there. Yeah. And this, this picture really doesn't do much justice as, uh, as Manfred mentioned, it's, um, eight of those, uh, systems. This, uh, these servers actually need to field engineers to, um, to pull out this, um, just this sled that's inside of there. So it's, uh, it's a pretty massive system. It takes some talent to not only install it, but just to maintain it as well. [00:26:40] Manfred Chua: It's 150 pounds, right? It's, uh, it's, it's a person. So. [00:26:47] Chris Carrero: So I think, so this, uh, concludes the presentation here. We're going to answer some of these questions that, that came through. So it looks like, so we have one, uh, one of the questions were. [00:27:01] Speaker 1: Yeah, one from Bradley there in the chat. [00:27:04] Chris Carrero: Yep. Are the OEMs working on producing liquid cooling ready machines that warranties would remain intact if LC is implemented? Um, yes, definitely. So, um, the, a lot of the, the OEMs, um, the tier ones specifically. So, uh, Dell has a version of this. It's a 9680, um, L, um, which is a liquid cooled version of it. Um, it is a single phase liquid and you guys now being on this presentation, know the difference between single and two phase. It's a water based solution. So it actually has water, um, or glycol, um, and water mixture running through the, um, running through the platform. So Dell does offer it. They do offer a warranty, um, on that system. Um, it obviously doesn't perform as well as the two phase solution. It doesn't remove heat the same way. And then you do have that risk of water on the inside, not just for that server, but other reasons in the data center. Um, so you have, um, super micro who's another, um, OEM or tier one OEM that has their solution. Um, park place is working with them. We actually do some deployments for them and, uh, in specific countries, um, for their, um, their water product. But there's, there's other OEMs. A lot of the ODMs have different options, um, to date, uh, unless Manfred corrects me. I think, uh, they're all single phase. I don't think you guys are working yet, um, with the OEMs, but I know there's, um, some work in progress to, to kind of get the approval, but, um, for the, for the warranty aspect. [00:28:44] Manfred Chua: Um, yeah, so we have, uh, we have a, so, so, so short answer is, uh, uh, as rock rack, uh, if it's not, um, if it's not like on their website yet, um, there will be a, uh, an AI server for H 200, uh, available. Um, I would say, I don't know what the right, the exact terminology, but like native from as rock rack, uh, with Zootacore's solution. Um, so it'll be, you know, fully warrantied straight from Nvidia. So that'll be, uh, available shortly if it's not already on the website and then, uh, Nvidia does have a, uh, what they call, uh, the partner cool program. And so it's think of it as if a customer is looking for Zootacore's, uh, cooling solution, then they would, uh, basically work with the OEM. So the OEM can elect to have our cooling solution, uh, uh, on their server. And, and, and basically it's just a, it's a call that they have to go do and it'll receive, uh, the full warranty from, from Nvidia. So there's a call cycle that needs to happen. Um, but that, that's, what's great about Nvidia anyway, is that they've, they've basically recognized that they're, they're not trying to manage it. Right. They're just saying, Hey, as long as it, it, it, uh, as long as the OEM is satisfied in terms of, you know, the product fits, you know, the quality is good enough for the OEM. And, and, and the, the system performs from a software development perspective, you know, like they, they, they have metrics that they want to uphold from, uh, uh, you know, from the, the CUDA perspective, then, uh, they're, they're, they're good to go. So, you know, we have, uh, the partner cool program. [00:30:43] Speaker ?: Excellent. All right. [00:30:44] Speaker 1: Well, why don't we move on to some of the questions we got in advance? Um, and I, I imagined Chris and Manfred will probably tag team these. I won't, I won't assign them to anybody, I guess. What would be the baseline of ISO or other regulations to introduce liquid cooling in the infrastructure? [00:31:02] Chris Carrero: Yeah. [00:31:03] Speaker ?: So it's interesting. [00:31:03] Chris Carrero: So certain countries, um, not so much in the United States yet. I think it's going to change probably soon as we start consuming more and more of the world's energy. I mean, a couple of years ago, we were at two and a half to 3%. I think we're at close to 5% now. And, uh, some of the projections are saying like 8% of the world's energy by 2030. [00:31:24] Speaker 1: Um, from, from data centers overall. [00:31:27] Chris Carrero: Yeah. From data centers overall, which is a massive, massive amount of power. Um, so countries like Germany, um, they enacted a, it's called the, uh, German energy efficiency act. So they're trying to reduce the energy footprint down, uh, 50%. Um, and that's assuming growth. So data centers are at least on track for growing 10% year over year. So now, even though the growth is still going to be coming in and nevermind any other new innovation or requirement for data centers, where we're looking at, um, trying to cut energy in half. Um, in, uh, in Germany and, uh, the EU, I believe has a, um, very similar regulatory package. That's that trying to cut it down, not so much 50%, but they're trying to reduce energy. Um, like just under 12%, uh, between now and 2030. Um, and then Manfred, I think you mentioned a little earlier, um, was it in France? Um, they basically, well, aside from being nuclear, which is a great thing. Um, you know, there's other, other things to worry about. Yeah. [00:32:34] Manfred Chua: And I think, uh, I mean, one France that you've got, you, you're, you're, you're, you're already probably good from a renewable energy perspective. So that's good. And then, um, I think depending on your jurisdiction and where you're operating in Europe, there's going to be other, uh, other, uh, I would say PUE regulations that are happening. I know that, uh, I believe the Netherlands is, uh, was, was looking at, uh, uh, uh, a, uh, PUE type, uh, data center thing. Uh, Ireland might be as well, given the fact that they're already in double digits from, uh, uh, energy consumption as well. [00:33:19] Speaker ?: All right. [00:33:20] Speaker 1: Uh, moving on is two phase cooling. Uh, it doesn't, doesn't delineate between, um, directed chip or immersion, but is two phase cooling considered now or more of something like in the roadmap for the next. Gen of, uh, GPUs. [00:33:36] Manfred Chua: Yeah, that's a great, that's a girl. I'll, I'll take that one, I guess. Um, so I would say, uh, that, I mean, of course I would say now, uh, and, and I think it really depends on the situation that you're looking at and what's happening right now is we're actually getting pulled into a lot of, uh, upfront data center design decisions, right? Because, uh, for example, depending on the, the, the type of data center design that, that our customers are looking at, uh, I'll give you an example. Uh, when we have a customer that is, is looking at, uh, designing a data center. That is a data center that is going to completely, uh, reduce or eliminate their water usage. Right? So not even having a primary water loop, but actually moving that, uh, primary water loop to, uh, uh, a two phase fluid, in which case, um, there are ripple effects to that decision. And one of them being, um, using our technology. Uh, and, and so, uh, they're doing that for a number of reasons. Uh, one of them, of course, is, uh, uh, they're in a particular geography that is prone to earthquakes. Uh, and so, uh, for them hydrophobia is top of the list because, again, you can't predict an earthquake. And, and so there is, uh, a, a strong consideration to remove that primary water loop completely. Uh, and so that's just an example of design, design decisions that are happening. Um, now when, when, so, so it's, it's, it's a much more complicated question because if, if you were to take a look at answering that question strictly from a thermal engineer perspective. Right. So if you are, um, a chip designer, thermal engineer looking at thermal design power, uh, the argument is probably gonna be you're about a year to a year and a half out thermal design power. We can probably get away with, uh, with air cooling actually, you know, for most of the roadmap that we've got. Um, however, I think when you take a look at the benefits that you're gonna see from a density perspective, especially as you're moving from. A, uh, uh, uh, you know, 10 to 15 to 20 megawatt data center to a 50 to 100 megawatt data center scale. And you need to densify and you need to start looking at your, not just your PUE, but your water usage effectiveness, your WUE and how dual phase direct to chip cooling, um, can drive the benefits of, of both of those. A, uh, then, then you start, uh, optimizing not just for thermal design power and thermal flux, but you're also, uh, optimizing for the data center level. So it's, it's a, it's a more holistic, uh, answer than just, you know, a pure power thing, uh, chip power, I mean. [00:36:55] Speaker 1: All right, Manfred, um, there's another, another question from Bradley. It said, you explained that the graphics on the previous slide were not perceivable at actual scale. How big are those red and blue pipes? [00:37:08] Chris Carrero: Oh, they're, um, I mean, I have a, I have a cold plate here. I'm not sure if I can share it. Oh yeah. Oh yeah. Good. It's looking at your background is going to make it hard to share. Yeah. [00:37:19] Speaker ?: Yeah. [00:37:19] Chris Carrero: The, um, it's probably like eighth inch, um, an eighth inch PEX pipe. I mean, they're, they're very skinny in comparison to, um, a single phase liquid cooling where you're trying to get a lot of volume of water to, to pass through the cold plate. Um, with this, like the blue side is very skinny. That's probably like almost maybe not sixteenth of an inch, but I would say in between like sixteenth and an eighth of an inch, um, for the inner, the inner diameter. And then the other side, the red is, um, that's where the gas comes out or where it changes phase. So it's a little bit thicker, but it's much, much smaller, um, than a traditional, um, like quarter inch or half inch hose that would be used in a, um, in a single phase. [00:38:11] Speaker 1: Alright, you know, one question that we got in advance and we, we get a lot when we're out at events and such is, you know, it's a big upfront cost. Where do I start? There's immersion, there's, um, direct to chip, there's rear door, there's, you know, the single and two phase of each of the immersion and direct to chip. Like where, where do people start? You know, when they're trying to figure out which one's right for them. [00:38:33] Chris Carrero: I would say you got to run a TCO. You got to try to figure out what you're trying to accomplish. Are you just trying to get one of these AI machines or a rack of these AI machines in the data center and you don't care too much about power consumption? Or are you trying to be sustainable? You're trying to not waste water. Um, you have access to water in the data center or you don't. There's a couple of questions that I would say need to get answered, um, first before making the decision. Um, and even, even if an OEM offers liquid cooling in their platform, do you want water on that system or in that system in the cabinet? Um, there's just, once those questions get answered, um, then you kind of run it through a TCO or a total cost of ownership, figure out what the ROI, um, how long it's going to take to, you know, either to pay for it back or, um, in some cases, um, you have no other choice. You know, you're going to put hot spots in the data center and if you do it otherwise, and it's going to cost, uh, more money in air conditioning or, um, or, or even co-location fees. Cause you can't, um, use it in a data center, but that's, that's where I would begin. I would try to answer some of the questions first, um, some of the major questions of what problem you're solving, uh, before hopping into a TCO. [00:39:55] Manfred Chua: Yeah, I would add to that in terms of, I think, uh, if, if you're in a greenfield, uh, situation, uh, I, I think the answer is probably much easier in terms of, I think it, it would be, um, optimizing for, uh, I would, I would, I would, I would say, you know, try and figure out what optimizes you for the most density of compute. Mm-hmm on what you're trying to do. Uh, I, and of course I would, um, you know, I would promote my technology because I think we could probably get there, but, um, I'm biased of course. Uh, but the reason why I say that though, is because, uh, when you optimize for your most, uh, dense compute environment, that is your highest ROI, right? You're, you're doing this because your compute is your moneymaker, right? I mean, you know, that, that's why we have data centers, right? Because, um, you're, you're, those are engines of revenue. Um, like what, uh, I think it's, it's been a year now since Jensen's keynotes, for example, have been saying there's a five to one ratio of ROI, right? Like for every dollar you put into an AI factory, it's $5 back. Now, I think at the scale he's talking about, I think he's saying that, uh, it's a billion dollars that you have to put in to get $5 billion back. So, you know, of course I hope all of you have a billion dollars to spend, but, you know, let, but, but still at, at, at the end of the day, I think it's what, what can get you the, the highest, um, ROI from a compute standpoint. And then I think the other one is also watching out the right, watching out through your regulations because, uh, there are, uh, so, so that's it. That's if you're greenfield now, if you're, if you're brownfield, then you've really got to watch out for, Hey, you're, you're probably going to be space constrained, or you're going to, you're going to have to deal with, okay. Can I route my primary water loop to that area, to that white space, or can I, you know, do I have the, the, whatever the, the, the weight, do I have a weight restriction? Do I have, you know, there's, there's all these other constraints that you have to think about if you're brownfield, um, because you're in an existing building or you've got, uh, other, other things to consider. But, you know, it all comes down to kind of what, what Chris was saying, right? There is a TCO model, but operationally, I think going liquid cooling should put you on the positive end of an ROI. Uh, and it's just a matter of, of, of trying to get and dial in that, that, um, that, that compute the highest level of compute that you've got. [00:42:55] Speaker ?: All right. [00:42:56] Speaker 1: I think we have time for one last question. This one's specific to the UK. So how would you see the regulations and policies in the UK impacting the immersion cooling adoption? [00:43:05] Chris Carrero: So the, I think I mentioned a little bit about the, um, the energy consumption. It's that 11, they're trying to squeeze, uh, 11 or reducing by 11.7 between now and 2030. I think it, uh, this was enacted in 2020. Um, this, there's been a big driver, uh, in the UK, um, for that reduction. Um, the liquid cooling technology of one of the things that I like is actually the heat reuse portion. Um, in, in the UK, they have facility for actually taking some of the heat from a data center and potentially selling it back or giving it to the community. So, um, in, uh, in ways of in the United States, we have a water tank. Uh, we generate our own heat, uh, in there in usually per household. Sometimes in buildings, it's shared, but in the UK, it's much more, um, there's much more infrastructure built where you can take a heat exchanger attached to a data center. Uh, potentially pushing that heat or that wasted energy back in and reselling. So, um, with those types of things, you have a much, um, better demand or there's more demand for some of these higher efficiency systems. Um, and then obviously the, um, that Energy Act, uh, for, for pulling, for reducing, uh, carbon footprint and just energy usage in general in the data center by 2030. There's a lot of, uh, a lot of need for, um, immersion or, um, or even just regular liquid cooling. It's not just immersion. [00:44:46] Speaker ?: All right. [00:44:47] Speaker 1: I think we'll, I think we'll end it there. Thanks everybody again for attending, um, any follow up questions. Um, you can, uh, reach out to your Park Place Technology sales rep if you have one. Um, otherwise, um, I think you can still drop something in the chat now. Um, or, you know, we have more information on our website, of course. So again, thanks everybody for joining us. We hope you learned something today. Um, and, uh, this was time well spent. Thank you very much. Thanks. Bye bye. [00:45:17] Manfred Chua: Thank you.