The REAL Cost of Data Centers: Infrastructure, Power, and Geopolitics — Part 1

[00:00:00] Speaker 1: and we're starting from the bottom of the stack the data centers don't worry this isn't gonna be yet another video about data centers yeah okay the video is about data centers as you might have guessed in the title but give me 20 minutes and i will show you why the most boring looking building in the world is the subject of an energy crisis a geopolitical contest and the largest capital investment cycle in modern history space ocean deserts and plains governments sovereign funds and the world's richest companies all racing to build data centers today we're launching something we've been preparing for a while a series called the business of ii because underneath every headline about models and chips and trillion dollar investments there is a web of interests capital and consequences that are not on the surface this series is the map and in every episode i will trace why countries and corporations are doing what they're doing who profits from it what consequences might follow and how it all connects across the global tech from a fiber cable on the ocean floor to the price you pay for a chatbot let's dive in this is a simplified ai stack at the very top are the apps that you use on a daily basis cloud deep seek midjourney all kinds of llm wrappers they run on your phone on your laptop in your browser or whatever screen it is you're holding underneath those apps sit the models the giant neural networks that were once trained on enormous amounts of compute and all of that compute in the chips gpus tpus npus the power the servers have to live somewhere physical which brings me to the gold mine of the ai stack the data centers now what is a data center imagine a building shell with a ton of racks and cabinets on racks there are servers storage unit and networking gear each server contains chips storage units are basically a bunch of ssds and yes it is simplified but i'm getting my point across they store things like databases or logs or models and lastly networking networking is a bunch of switches that connect servers to each other yes they do a bunch of other things too but for simplicity they connect things into a network this is the basic combo now multiply this combo by n and you've got yourself a massive building shell with thousands of servers and with a ton of metal chips in those servers and this beast starts functioning and what happens when it starts functioning it heats up and when you've got a building the size of a football field that heats up you probably realize what else needs to happen to keep it running cooling electricity water power and land and all of these things are commodities now zoom into the commodity that is probably the hardest one to get in this entire chain electricity now i want to talk about electricity in detail because there's a lot of chatter about how we're facing the energy crisis how utility bills are going to go up how electricity is becoming the new oil but what exactly does it mean why is it becoming the new oil can't we just produce more of it so let's break it down say we have a data center which has racks which has servers which has chips each server needs a certain amount of energy to run Why? Because servers contain chips. And every modern chip, a CPU, a GPU, or a TPU, contains billions of transistors. Each transistor is a microscopic gate, so to say. If you add voltage, the current flows. If you remove it, it stops. When your chip, a GPU for example, is doing AI inference, it's switching billions of those gates on and off. And to do that, it needs a power supply. Now what I just said happens inside one chip. Now multiply this by the number of chips in a server, and then by a number of servers in the building. The amount of electricity that one server consumes per month is equal to what a typical one bedroom apartment needs. But remember, you have a ton of servers in a data center, depending on how big that data center is. But this is actually secondary. The most important thing is that when a data center draws one unit of electricity, that entire unit of electricity converts to heat. So if you have the energy that a data center needs to run, all of that energy, not most of it, all of it, will convert to heat. The peculiar thing is that the data that gets produced on the servers is a byproduct of this process. The data itself needs less than 0.1% of that energy conversion. The rest is pure heat. And that heat has to be offset. Otherwise, the hardware will literally cook itself, the performance will crash, and the whole data center turns into a massive box of extremely expensive and useless metal. To offset the heat, you have to cool off your data center. And cooling takes about 40% of total electricity usage because every joule that moves through a chip has to be physically expelled from the building. When you look at it at scale, this means that you're basically looking at a power plant's worth of power consumption just to keep the data center functioning. If you cast a net over all data centers worldwide today and add up how much power they consume, you will end up with a figure that's roughly the entire electricity consumption of Germany or France, both of which are highly industrial nations. By 2030, on current trajectories, data center consumption will be comparable to India's. And mind you, India is the world's third largest electricity user. In Ireland, data centers already took 20% of national power consumption, which triggered moratorium on new data centers in Dublin. So you may listen to this and think, okay, so we need more electricity. We always need more of something. Why is this a problem? This is a problem because unlike software, for example, you can just magically deploy more of it. Electricity doesn't exist naturally at scale. It must be manufactured. It can be manufactured from various sources: natural gas, nuclear fuel, water, wind, solar power. But to convert those sources into electricity, you need a lot of rotational motion and turbines. If the source is natural gas, you need a power plant that has a lot of rotational power and turbines. If it's nuclear fuel, you need a nuclear plant. If it's water, a hydropower plant. If it's solar, a solar plant. Meaning that when you read headlines about AI needing more electricity or needing more data centers, think more power plants and more of the grid. Now, what about the grid? Electricity is like a bloodstream that runs through a system and that system is the grid. The grid has to stay in balance. Every watt that is generated has to be matched by watt used at any moment. For example, when a power plant pushes electricity into the wires, that electricity has to be used right away by homes, factories, businesses, data centers, you name it. In that same moment. If power plant produces more or less than people are using, even for a short time, the power system starts to fail. The European grid, for example, maintains its balance at 50 Hertz and the US grid at 60 Hertz. But if people suddenly try to pull more power than the grid can deliver, the system starts protecting itself. It would first turn off some parts of the network to reduce load. And if that still isn't enough, more and more plants and lines drop out of this chain reaction and you end up with a full blackout. The scalability of the grid is the core reason why data centers cannot scale indefinitely. The grid does not have a buffer. There's no magic warehouse of electricity that you can open and deploy when demand spikes. So when you're building a new data center, you have to make sure that when you connect it to the grid, the grid can maintain the demand. But the demand is growing at an unimaginable speed. A hyperscaler like Google or AWS has the budget to throw on a new data center. That's not an issue. They can buy the hardware, servers, racks, chips, cooling units. But a new power plant takes around 10 years to build. A new voltage transmission line takes eight years. A line to get a new generator connected to the grid has the wait time of five years. And the US electricity grid was never built for this volume because for decades, electricity demand was generally flat. And when traditional data centers peaked during the dot-com boom, the demand was still variable. Servers were often idle. Utilization was uneven and load could be predicted. In contrast, AI data centers run at near 100% utilization. All of this feeds into a global shift in the price of electricity. And this shift is not reversible at this point. Last year, electricity prices rose across the US with AI infra being the primary driver. The cost of the grid upgrades is already in your utility bills. The International Monetary Fund modeled two scenarios. If supply is able to respond fast, AI's energy demand would only cause a small price increase. But again, let's recall what is meant by the supply in this case. Power plants and the expansion of the grid capacity, which will take several years. And if it doesn't, and the evidence so far suggests that it won't, the price surge can be steep enough to slow down the growth of the AI industry. In other words, there is a hard ceiling as to how far AI can grow unless the US comes up with something completely radical. This is the reason why you hear electricity and energy crisis everywhere. Because electricity is the cornerstone and the bane of data centers. So we've got a dilemma. The US has huge ambitions around AI, but it can't keep stacking all of its load on its own grid. But hey, China's stepping on your and you can't afford to stop. So you look around and ask, well, who else has the power, the land and politics to build more? And now let's talk geopolitics. Under President Trump, the US made a decision to deploy private and foreign capital to build data centers as rapidly as possible. China at the same time poured a ton of investment into data centers three years ago. But by 2025, 80% of new computing capacity sits unused. Why? Why in the world where Meta funds the construction of natural gas plants to have more compute, China has it but doesn't use it. Two reasons. China built for the wrong purpose. And the US did everything to make sure that they couldn't fix it. To understand what this means, you need to know the difference between training and inference. Training is the process of teaching a model. You take a massive data set, trillions of words, images, code, and you train a neural network until the model gets good at predicting what comes next. This happens generally only once or a handful of times for future iterations. Training requires enormous number of brutally powerful GPU chips. But what's more important here is that it doesn't matter if the training takes two or three weeks as long as it finishes. In other words, training is not latency sensitive. Inference is the model running, answering your questions, generating content, summarizing emails. The model reads your input and generates an output. And inference is the opposite of training. It is extremely latency sensitive. For example, if Claude takes 30 seconds to respond, you're not going to pay for it. That's latency. Now, let's talk about what China built and when. February 2022, before ChadGPT, before the AI boom. China said, "Our population and our economy is mostly concentrated on the East Coast. But cheap electricity and land are in the West. We're going to move the compute west, route the traffic east, and we get cheap and reliable AI infra built and controlled by us." Then ChadGPT launched in November 2022. The government looked at it and said, "AI computing is now a national priority. We need tech investments and we need to build data centers." And so they planned to build at least 500 of them. However, the bet they made was that the data centers would be used for AI training, not inference. They built for raw power density. They filled them with chips, mostly NVIDIA H800s and some Huawei Ascent cards and a lot of CPUs. But the problem is that the industry rapidly changed under their feet. By 2024, the AI industry completely pivoted. Training frontier model is something only three or four companies in the entire world do at a given time. And it's most likely OpenAI, Anthropic, Google DeepMind and maybe Meta. The vast majority of AI compute demand is now inference. Meaning using models that already exist and apply them to actual problems. China's data centers were built for training. But the demand is in the inference. And those are not interchangeable. A remote facility in Western China with poor latency to Shanghai cannot serve real user requests rapidly. No matter how many GPUs it has. Pure fiber distances and routing means that your latency is going to run high. Now, does this mean that China is losing the race? No, hell no. Yes, they are behind on frontier models. Their domestic chips are less powerful than American. But China holds an edge in electricity and land. In 2024 alone, they added more electricity generation capacity than the US has in an entire decade. If the AI race shifts, which it very much can, to who can scale inference cheaply and at massive volume as opposed to who produces the biggest model. China has perhaps the most superior energy infrastructure in the world because it is very efficient and relatively cheap. But US and China aren't the only kids on the block. There is a third geography that has land, cheap energy, sits at the crossroads of Europe, Asia and Africa, and is sitting on enough oil money to buy their way in. The Gulf. And the Gulf wants in. See, the Gulf and Saudi Arabia in particular is very well aware that they are a candidate for the Dutch disease. Oil is a finite and politically problematic asset. The crown prince of Saudi Arabia, Mohammed bin Salman, centered his agenda around repositioning of the kingdom as a tech and AI hub. Saudi Arabia doesn't manufacture batteries or chips at scale. They don't have a deep domestic talent pool for tech on the level of the US or China. But what they do have is the land, energy and money. At the same time, the interesting thing about the Gulf is that it is not only in their interest to partner with the Americans or Chinese. So why does the US need the Gulf? The US is simply running out of room. And the problem is in the land. America is a gigantic country. They don't have an issue with land. What they do have an issue with is electricity and grid capacity. Like I said at the beginning of the video, the US cannot just build new data centers overnight. Because the grid cannot absorb so much load at once. The second problem is cooling. The US builds most of its data centers in the states where they've got plenty of unused land. Texas, Arizona, Nevada and Southern California. They need crazy resources to pull enough water inland to cool the data centers off. And number three, the electricity costs in the US are four times higher than in the Gulf. In other words, the US is hitting a ceiling. And the Gulf is one of the few places on Earth with enough money to afford investments into AI infra where the ceiling doesn't exist. The Gulf has cheap and scalable energy. They invest heavily in solar and nuclear plants, green hydrogen and conventional gas. Land. And not just any land. They're building data centers in places with fantastic proximity to resources. Places like Abu Dhabi and Riyadh are surrounded by open desert, but they're just as close to existing utilities, fiber station and transport links as they are to the desert. This is the opposite of the US problem where every acre of land near existing grid and fiber infrastructure is already spoken for or used. It's expensive and it is opposed by residents. In Abu Dhabi, for example, the government is the land owner, the grid operator and the authority. If the government wants a data center, they are the ones approving it. And lastly, look at the globe. The Gulf sits almost exactly between Europe, South and Southeast Asia and East Africa. And it is connected to all of them by multiple submarine fiber cable routes. We will make a separate episode on the cable routes. And those cable routes land in the Gulf specifically. A data center in Abu Dhabi can serve users in Mumbai, London, Nairobi and Singapore with competitive latency. This is the dream come true for the US because no single US data center can do all four simultaneously. The Gulf can serve everyone between the Atlantic and the Pacific at lower latency and lower costs than routing traffic through US data centers. The only problem that the Gulf has is that it can't play the AI game alone because they need those American chips and they need Chinese expertise in AI. So they were, at a certain point, rationally playing both sides. China got elbow deep into the Emirati and Saudi tech scene a long time ago. Between 2015 and 2022, Huawei built large portions of 4G and early 5G telecoms for both the UAE and Saudi Arabia. Bydance, Tencent, Alibaba Cloud all have Middle Eastern operations and have been hiring very actively across the region. G42, Abu Dhabi's most important AI company in the region, bought Huawei servers and telecom gear, ran parts of its stack on Chinese hardware, partnered with Chinese firms on genomics and health data, held about $100 million worth of TikTok shares, and employed Chinese engineers who helped build and operate the infrastructure on Huawei's stack. When the US started watching G42's ties with Chinese, they started raising their eyebrows. The US has been historically reluctant to sell advanced AI chips to the Gulf because they had concerns that those would be resold to China. But the tension has gotten so high that the Biden administration went directly to the UAE and gave them an ultimatum. Either we blacklist G42 and cut them off from all US technology, or you sever all ties with the Chinese. And G42 picked the US. Not having access to US chips, US cloud, US partnerships is a death sentence to any company trying to build frontier AI infrastructure. G42 divested its Chinese partnerships, removed Huawei's equipment, migrated to Microsoft Azure, and accepted US security protocols. This is why the Gulf is such an active participant in the US data center build out. They invest their money in exchange for political goodwill. This is why the US is so actively involved in the development of AI infra in the Middle East, despite having a history of very difficult relationships in the region. The US holds the world's dominance in chips and tech companies, but it is running into grid and political ceilings. China has the power plants and the land, but they misallocated their first big bet and is now trapped under export controls. And the Gulf is trying to turn their resources into an industry that can get them off the oil hook. So did China give up? No, they haven't. And this is another fascinating story, but more on this in the next episode. This is the end of part one of the data center episode. In part two, we will get into the money behind data centers. Who gets paid when someone spins up the new cloud? How billions move from sovereign funds to Nvidia? And why the real margins are in the boring layers like land, concrete, copper, and power? We hope you enjoyed this. Please let us know what you think in the comments. Until next time. Bye.