2025 OCP EMEA Keynote - Data Center Evolution Powering the AI Revolution Sustainably (Google)

[00:00:00] Please welcome Google Principal Engineer, Madhu Iyengar. [00:00:08] Hello, thank you to the OCP community for inviting us to talk about this topic. [00:00:23] We are really excited to be here. [00:00:25] We've all heard about the explosive growth of AI and underpining this incredible growth [00:00:33] in AI is the data center physical infrastructure. [00:00:36] That's the power, cooling and mechanical systems that create the infrastructure for this AI [00:00:42] on the hardware side. [00:00:45] So before we jump into the physical infrastructure, we're going to talk a little bit about how [00:00:50] AI is different because AI is very different from traditional compute. [00:00:56] So Google invented the transformer in 2017, which led to the large language models. [00:01:02] Every six months, every year, there's been more and more models that are released. [00:01:10] And these models are computationally higher performance and require higher physical infrastructure [00:01:18] in terms of cooling power and space. [00:01:21] Most recently, we released a Gemini 2.5. [00:01:25] So as you can imagine, as there's this explosive growth in AI on the computational side, [00:01:31] there's also a major growth and huge demand on the physical infrastructure side. [00:01:38] So what you're seeing here is a time series plot with data center power on the y-axis [00:01:46] and time on the x-axis. [00:01:48] This is for traditional compute for a traditional workload. [00:01:52] These workloads are typically uncorrelated. [00:01:54] There's large number of different workloads running at the data center. [00:01:58] And the variations that you see are fairly nominal. [00:02:04] What you're seeing here now is AI, and you can see it's different. [00:02:09] That's because when you look at large-scale batch synchronous training, [00:02:13] you have tens of thousands of chips that go from idle power all the way to a higher power state, [00:02:20] and then they drop back down. [00:02:22] You can have one large workload running in the whole data center. [00:02:27] In some cases, you can have a large workload running across multiple data centers. [00:02:31] So when you look at this side by side, [00:02:35] you can clearly see there's a huge difference between AI/ML and traditional compute. [00:02:40] This kind of difference in terms of how the workloads operate [00:02:44] and what kind of burdens and challenges that impose on the physical infrastructure [00:02:50] require new ways of doing things. [00:02:53] There's clearly quality, reliability, and physical infrastructure implications [00:02:59] from the chip all the way to the grid. [00:03:02] Another area that's very different is rack power. [00:03:06] The rack power for traditional computers changed pretty modestly over time. [00:03:11] But in the same time duration, the rack power for AI has grown very, very dramatically. [00:03:20] So this kind of power and power density also means that you need new ways of doing power [00:03:25] and cooling and physical infrastructure. [00:03:27] Google has a long history of working with OCP. [00:03:33] It is a relationship and a collaboration we really cherish. [00:03:37] We worked on data center facilities, sustainability, [00:03:41] hardware management, security, storage, and many other areas. [00:03:47] We expect to be working even more closely with OCP on AI/ML going forward. [00:03:53] So let's talk about sustainability, quality, and reliability. [00:04:01] Google has a sustainability goal of going net zero by 2030 in our operations and value chain. [00:04:08] We are leading the data center facilities sustainability space within OCP. [00:04:15] We have also co-founded the net zero incubation hub. [00:04:18] There are two initiatives that I want to point to. [00:04:23] One is along with our hyperscaler partners, there's been a lot of advancement in green concrete. [00:04:31] A second item I want to provide is clean backup. [00:04:35] This is backup energy to replace diesel generators. [00:04:40] Both of these topics have presentations tomorrow in case you want to learn more. [00:04:46] Coming back, we talked about how AI/ML workloads are very different from traditional compute. [00:04:53] This plot is a similar plot, but it shows chip temperatures. [00:04:57] As you can see, if the power is swinging from a low state to a high state and back down to a low state, [00:05:03] the temperature is going to follow the swings. [00:05:05] That kind of large and frequent temperature swings have pretty strong physical consequences. [00:05:16] At Google, we're working on a full stack optimization because rather than only adapting and reacting to the workload, [00:05:24] we also want to influence the workload to mitigate these issues. [00:05:30] And look at the core design of the whole stack, hardware and software. [00:05:34] ML power footprint is going to outgrow traditional compute in the coming years. [00:05:42] The issues and challenges that I talked about around power and cooling and AI workloads, [00:05:49] the large number of frequent swings, this is not unique to Google. [00:05:55] Other hyperscalers and cloud providers face the same issues. [00:05:59] So we really think there is a huge opportunity here to work across multiple stakeholders, [00:06:04] utility providers, cooling and power equipment suppliers, [00:06:09] hardware and software designers, ML workload developers to really forge forward. [00:06:16] So this keynote is really a call for action for greater engagement with OCP and the AI ML community. [00:06:25] Now let's talk about power delivery. [00:06:28] Google championed 48 volts DC in 2016. [00:06:32] And at the OCP summit in 2016, the community responded with a lot of collaboration around 48 volts DC. [00:06:40] This is 48 volts DC compared to what was then the original 12 volts DC. [00:06:46] And it created a rack infrastructure that was more efficient. [00:06:51] And it served us very well between 10 and 100 kilowatts of rack for the compute platforms. [00:06:57] But we think that's going to change. [00:06:59] And here's why. [00:07:01] There's two very clear trends in AI systems. [00:07:06] One is we expect rack powers of greater than 500 kilowatts by 2030. [00:07:11] The second one is there is a lot of competition for every millimeter cube of space inside the rack for XP use. [00:07:19] Which means you're going to have to push out some of the supporting infrastructure inside the rack to be outside the rack. [00:07:26] So because of these two trends, the first one, the 500 kilowatt plus in a rack, you're going to need higher efficiency power delivery. [00:07:34] And the second trend of putting in more and more XPU chips inside the rack, that's going to push out some of the infrastructure. [00:07:43] So you need something different. [00:07:45] We're really pleased to announce the plus minus 400 volts DC. [00:07:50] The first embodiment of this plus minus 400 volts DC is a standalone rack that's adjacent or close to the IT rack. [00:08:01] It receives AC power, traditional AC power from the data center, converts it and supplies plus minus 400 volts DC to the IT rack. [00:08:11] So that's the first embodiment shown. [00:08:13] You can see it there. [00:08:16] The reason we have chosen a plus minus 400 volts DC is because we can leverage a lot of the supply ecosystem from electrical vehicles. [00:08:24] This architecture is expected to be about 3% more energy efficiency, energy efficient end to end. [00:08:33] The energy efficiency comes from being able to operate at a higher voltage, which reduces the current and reduces the I2R power losses. [00:08:41] Going forward, we expect that we would probably be looking at supplying high-voltage DC directly to the rack without this intermediate sidecar rack. [00:08:52] At last OCP summit, Google, Meta, and Microsoft all talked about divisions for plus minus 400 volts DC. [00:09:02] We're really happy that we're working together now since the summit. [00:09:06] And we're going to jointly be working under the project named Mount Diablo and releasing a 0.5 spec at OCP next month. [00:09:14] Moving on to cooling, we've talked about chip powers going from 100 watts to 1000 watts to several kilowatts. [00:09:24] This requires a new way of cooling. [00:09:28] Liquid cooling has emerged as the cooling methodology of choice, at least in the near term for AI. [00:09:36] Water is a much better coolant than air. [00:09:42] The liquid transport capabilities are 4,000 times that of air. [00:09:47] The thermal conductivity is 30 times higher than air. [00:09:51] And so, we are looking at liquid cooling in the coming few years as a means of cooling AI/ML systems. [00:10:00] Google introduced liquid cooling for TPU version 3 in 2018. [00:10:05] And then, subsequently, between version 3, version 4, all the way to version 7, called IronWood, we have refined this technology. [00:10:16] Just as an example, for TPU version 3, the server was half the geometric volume of the air-cooled predecessor version 2. [00:10:25] And the super pod or the supercomputer was 4 times bigger. [00:10:29] That gives you a sense of what is possible in terms of power density and computational performance, which requires larger models and larger physical infrastructure that gets enabled with liquid cooling. [00:10:43] We have a global footprint. [00:10:48] And about half the data centers globally have liquid cooling enabled and liquid cooling deployed. [00:10:52] Our implementation uses a coolant distribution rack, the CDU. [00:10:58] And it has a lot of redundant features, including the UPS, uninterruptible power supply, to allow for much higher availability. [00:11:08] The coolant distribution unit separates the IT rack loop from the CDU loop. [00:11:16] And it provides for a much higher performance liquid cooling loop. [00:11:22] This is an embodiment of our fourth generation of liquid cooling. [00:11:29] It's called Project Dishutes. [00:11:31] Google has deployed about a gigawatt of liquid cooling across 2,000 TPU pods at an unprecedented and remarkable uptime of 99.999%. [00:11:43] So we really think we need to enable the industry. [00:11:47] And to do that, we're going to be releasing Dishutes next version, which is version 5 spec. [00:11:56] And it's going to be the design, the spec, as well as our best practices. [00:12:03] So right before I finish, I want to just show you this animation of this design that's in flight that we're going to be releasing to OCP. [00:12:12] And I want to be, I want to talk about how excited we are to engage with OCP on sustainability, on power delivery. [00:12:23] And hope that the community will embrace our project issues. [00:12:28] And we can learn from the community as we jointly take this AI/ML physical infrastructure forward. [00:12:35] Thank you.