Why AI Data Centers Are Becoming Impossible to Build

[00:00:00] Speaker 1: In early 2023, Meta tore down a data center it had half-built in Temple, Texas. You'd assume something went wrong on the job site. But the building was out of date before it was even finished, because the AI chips it was meant to hold had already outgrown it. It was one of the first signs of the strangest problem in the AI race. The richest companies on the planet can afford all the chips they want, and they still can't get the buildings, the cooling, and the power ready fast enough to switch them on. Meta is the company to follow here, because it has hit every wall in this story first. Its goal sounds simple: build AI supercomputers measured in gigawatts, each one drawing as much electricity as a city, and build them fast enough to keep up with NVIDIA's chips. The obstacle is everything those chips have to plug into, from the cooling in the walls to the power lines outside. Temple is where that obstacle first showed up in steel and concrete. Satellite images of the site, out on the open land between Austin and Dallas, tell the story frame by frame. Early that year, a long, H-shaped Meta data center stands there, half-finished, built the same way Meta had built its server farms for years. A couple of months later, the building is gone, and the ground has been scraped clear. Publicly, Meta talked about a pause and a redesign. Shortly before the teardown, it had stopped work on the Temple site, put building projects in Idaho and Alabama on hold, and canceled an expansion in Odense, Denmark outright. Its explanation was that running AI at scale needs a different kind of data center from the ones it had been building to serve Facebook and Instagram. The satellite pictures show how far that redesign went. The problem was heat. The old design was planned around ordinary server chips, and GPU chips pull far more power and throw off far more heat. Meta's engineers said that forced a full rethink, with liquid cooling brought right up to the chips. The networking had to change too, because the GPU chips in an AI cluster trade data with each other constantly. Meta's own planning documents, reported at the time, called for 24 to 32 times the networking capacity of its old halls. Finishing the old structure would have meant opening a data center that was obsolete on day one. So, it came down. Work restarted on the new design later that year. Meta claimed the new buildings would cost 31% less than the old ones, and go up much faster. That's Meta's own number, so treat it as a target until someone outside the company checks it. The rebuild took a while. This past July, the Temple data center finally started serving traffic, more than four years after Meta first shared plans for it. Meta says it's the first building to go live on its new AI design, cooled by a closed liquid loop that the company says needs no water for most of the year. Four years is a long time in this business. In that window, NVIDIA brought out two whole generations of data center GPUs, Blackwell and then Blackwell Ultra, and had just started shipping a third, Vera Rubin. And to see why the buildings can't just move faster, you have to look inside a single steel cabinet. That cabinet is called a rack, the tall frame the servers slide into. And a classic cloud data center is basically a warehouse full of them. Each rack runs its own small jobs. A website here, an email inbox there, and none of them care much what the neighbors are doing. An AI training hall works the opposite way. The whole floor is wired together to act like one enormous computer, with tens of thousands of GPUs working on the same model and constantly passing results back and forth. Inside each Blackwell rack, 72 GPUs are tied together over thousands of copper cables, so the cabinet behaves almost like one giant chip. That changes what every rack has to carry. Across the industry, the average rack in a typical data center still draws under 8 kW. NVIDIA's Blackwell racks, the systems going into AI halls right now, each draw around 120 kW or more. Same cabinet, about 15 times the power. That jump explains Temple better than anything else. Every cable, breaker, pipe, and air handler in a data center is sized for a certain load per rack. When that load jumps 15-fold, none of that sizing holds anymore. And no simple upgrade fixes it. That's why META started over in Temple. 600 kW is the figure NVIDIA has put on Kyber, the rack design it plans to ship in 2027. Every generation packs more GPUs and more power into the cabinet. Kyber is about five times the power of a Blackwell rack, squeezed into the same kind of cabinet. And NVIDIA has already talked about racks that need a full megawatt after that. Those are the racks the halls going up today will have to be ready for. That leaves every builder in a trap. A large data center takes years to design, permit, and build. But NVIDIA now brings out a new rack generation roughly every year. So the concrete going in today is for chips that don't exist yet. Guess too low, and you get Temple again, a building the chips outgrow before it opens. So builders now lean toward oversizing, even though it means paying for power and cooling that could sit unused for years. Many operators now hedge. They put in the power and cooling first, and fill the halls with dense racks later, adding them as the chips arrive. And filling those halls means breaking the oldest rule in the building, the one that says air can keep everything cool. Every kilowatt that goes into a rack comes back out as heat. And for decades, air carried it away. Cold air goes in the front, fans push it across the chips, and hot air comes out the back. And the trouble is that air carries heat poorly. Somewhere around 30 to 40 kilowatts per rack, you physically can't push enough of it through a cabinet without the fans themselves eating a big slice of the power. Every Blackwell rack is far past that line. So the industry turned to liquid. In a liquid-cooled rack, a metal cold plate sits right on top of each GPU, with coolant flowing through tiny channels inside it, carrying the heat out of the hall to big coolers outside. Water can carry far more heat than air, which is why this one change makes the dense racks possible at all. It's the system meta-built into Temple. Liquid brings its own risks, though. If the pumps stop while the racks are at full load, the chips can overheat within seconds. So the cooling needs its own backup pumps and backup power, just like the servers do. Then NVIDIA went further. A single Vera Rubin rack can draw around 200 kilowatts. A household space heater on full blast draws about 1.5 kilowatts. So that single cabinet burns as much power as well over 100 of them. NVIDIA cools every compute tray in that rack with liquid. And the trays have no hoses hanging off them, and no fans at all. Think about what that does to a building designed around air. The giant air handlers, the carefully planned hot and cold aisles, the whole layout a classic data center was built around, now has far less to do. In its place, you need pipes, pumps, heat exchangers, and leak sensors running under every row. And how do you even get that much power into one cabinet? Inside today's racks, power travels at low voltage. And pushing hundreds of kilowatts at low voltage takes thick, heavy bars of copper. So NVIDIA is moving the industry to 800 volt direct current, which is nearly seven times the voltage at a North American wall socket. The power gets converted from the grid once, then travels at that higher voltage, so far less copper can carry the same load. NVIDIA says that system reaches full production alongside the Kyber racks. That's two redesigns at once, the cooling and the wiring, every time NVIDIA moves up a generation. They're expensive, and every new AI hall now needs them. The equipment they call for is where the real waiting starts. Start at the edge of the property, where the grid comes in. Electricity arrives over high voltage lines, and before a data center can use it, huge transformers have to step that voltage down. A large power transformer is custom-built, weighs hundreds of tons, and is wound from specialized steel and copper. Each one is built for a specific site, tested at the factory, and then hauled out on special rail cars because it's too heavy for most roads. Today, the average wait for a standard one is about 128 weeks. That's roughly two and a half years, nearly as long as it took META to finish Temple after work restarted. The biggest units can take four years or more, and prices have jumped by roughly 80% in just a few years. It isn't only transformers, either. Switchgear, the heavy switching equipment that routes and cuts power inside the building, is stuck in the same queue, along with circuit breakers, because utilities, factories, and AI builders are all ordering from the same small group of suppliers at once. The suppliers are responding. This June, Hitachi Energy broke ground in South Boston, Virginia, on a big new factory for these giant transformers. But a plant that's only now breaking ground, can't ship anything to the halls going up today. This is where the problem stops being theoretical, because the chips are now arriving faster than the rooms to put them in. Late last year, Microsoft's chief executive, Satya Nadella, made an unusual admission on an investing podcast. He said Microsoft's biggest issue had become power, and getting buildings finished close enough to that power. His warning was that Microsoft might be left holding NVIDIA chips it had no place to switch on, because it was short of what he called "warm shells," finished buildings with the power and cooling ready to go. Think about who was saying that. Microsoft is one of NVIDIA's biggest customers, with every reason to talk up its AI build-out, and its boss was telling the world the limit was buildings and electricity. He even said he'd rather not buy too much of any one generation of NVIDIA chips, because the next one shows up so quickly. For years, the big worry in AI was whether TSMC could make enough chips. Nadella was adding a new worry to that list, one that sits at the substation. Microsoft's own answer is Fairwater, a campus in Mount Pleasant, Wisconsin, built to hold hundreds of thousands of NVIDIA GPUs. Microsoft stacked its racks on two floors, so the GPUs could sit closer together, and the cables between them could stay short. Beneath the site run about 190 kilometers of underground cable, more than four-and-a-half marathons laid end-to-end. It's cooled by a closed liquid loop that Microsoft says is backed by the second-largest water-cooled chiller plant on the planet. And every bit of it still depends on power from outside the fence, which is where this bottleneck gets a lot bigger. A grid connection is slower still, because it depends on power lines, substations, and new power plants, and those take years of planning and permits. In Northern Virginia, the biggest data center market in the world, the local utility, Dominion Energy, now has requests from large customers adding up to nearly three times the most power its whole system has ever delivered at once. A big new data center there can wait as long as seven years for its connection. Virginia's regulators have already created a separate electricity rate for the biggest users, aimed at keeping the cost of new lines off everyone else's bill. So where do builders go when Northern Virginia is full? Texas was supposed to be the escape route. It has open land, plenty of gas and wind, and a grid of its own. So the industry piled in, Meta included. By this summer, the line of requests to plug into the Texas grid had reached around 474 gigawatts, nearly all of it from data centers. That's more than five times the most electricity the entire Texas grid has ever delivered at one moment. Plenty of those requests are speculative. The same developer reserving power at several sites and planning to build at only one. Texas had already passed a law making big new users pay $100,000 just to apply and own up to any duplicate requests. Even so, a small slice of that line would swamp the grid. And then Texas did something that would have sounded unthinkable when Meta first broke ground in Temple. In early August, Governor Greg Abbott ordered a full audit of every data center waiting to connect to the Texas grid, and no new data center could move forward until it was done. The grid operator paused approvals to switch on new data centers and crypto mines while it worked through the list. Just days earlier, when the grid operator sorted that line, only about 4 gigawatts in every 10 came from projects that had cleared its first eligibility check. Now look at the timing. Meta's Temple Building, the one torn down and rebuilt for the AI era, started serving traffic in late July. On the same day, the Texas grid set its all-time record for demand. Less than two weeks later, the state it sits in froze the line of data centers waiting behind it. One of the most building-friendly places in America had decided the power question came first. And plenty of builders were fine with that. At least 43 companies building data centers publicly backed the audit, and clearing the speculative requests out could move real projects up the line. If you're getting something out of this, subscribe. Tomorrow Engineered covers the machines, the factories, and the companies racing to build the world's most advanced chips. And who's really ahead? 1.2 gigawatts is how much power Elon Musk's AI company, XAI, now has permission to generate from gas turbines on a single site in Mississippi. All of it for its own supercomputers. That's enough electricity for roughly 900,000 American homes. While the grid queues kept growing, XAI decided not to stand in them at all. For its Colossus II supercomputer in Memphis, Tennessee, XAI bought the site of an old power plant just across the state line in South Haven, Mississippi, and started filling it with gas turbines, beginning with portable units it could truck in. Mississippi regulators then approved a permit for 41 turbines there. At its first Memphis site, XAI ran temporary turbines while it waited for the grid. And now that its substations are online, it has been taking some of them away. It hasn't been smooth. Local environmental and civil rights groups accused XAI of running turbines before it had the right permits. And the fight over air quality in South Memphis is still going. Shelby County has since granted a permit for turbines at the Memphis site, and that permit is under appeal. But the logic is easy to follow. A turbine you can buy and truck in can be running long before a grid connection that might take years. Except, the turbines are running out too. GE Vernova, one of the world's biggest turbine makers, is sold out of gas turbines through 2028, and it expects data centers to drive about a third of its American gas power deals. A builder who decides today that it wants its own gas plant joins a line stretching toward the end of the decade. So even going off the grid now means waiting years for equipment. So the biggest players are locking up power years ahead, any way they can. Microsoft went back to a name almost everyone knows. It signed a long-term deal that's bringing a shut-down reactor at Three Mile Island in Pennsylvania back to life. Now renamed the Crane Clean Energy Center, with power aimed for next year. Once it restarts, the reactor is expected to put about 835 megawatts back on the grid. Google went a step further and bought an energy developer outright. Its parent company, Alphabet, paid $4.75 billion in cash for Intersect, a company that builds power plants, solar farms, and batteries right alongside data centers, starting with a shared site in Haskell County, Texas. That's about four times what Meta says it spent on the entire Temple data center, just to own the power side of the business. Meta took the idea further than anyone. In New Albany, Ohio, outside Columbus, it's bringing up Prometheus, its first gigawatt-class AI cluster, with part of it already running. By outside estimates, the rebuilt Temple site runs on roughly 150 megawatts of computing power. Prometheus is designed for more than six times that. To get it built faster, Meta put part of it inside giant, weatherproof tents, aluminum frames wrapped in fabric that go up far quicker than a concrete hall. Mark Zuckerberg has said the tents are there for speed. Meta isn't relying on the grid alone, either. Filings show the site has its own power generation yard, tied to the data halls by a wide utility corridor. And Ohio regulators approved two dedicated natural gas plants to feed the campus directly. Meta has also lined up nuclear power from Vistra, TerraPower, and Oklo to back it, some of it from reactors that haven't been built yet. Five gigawatts of computing power is what Meta wants from Hyperion, a campus it's building in Richland Parish, out in the rural northeast of Louisiana. The other thing that's been built in the U.S. is a big part of Manhattan. To pay for it, Meta set up a $27 billion joint venture with the investment firm Blue Owl Capital, which keeps much of the debt off Meta's own books. To power it, Meta didn't just sign an electricity contract. Regulators had already approved new gas plants for the site. Then, this March, Meta and the utility Entergy announced a much bigger deal. Hyperion is now set to be served by 10 gas power plants, more than 7 gigawatts of capacity, for a single customer. That's more than half of all the power Entergy currently generates across the entire state of Louisiana. It's a fleet of power plants big enough to reshape a state's grid, and the new ones still need sign-off from Louisiana's regulators. That review is where the real fight is, over who pays if plants built to run for decades outlive the data center they were built for. Meta and Entergy say Meta will cover its own costs. Put it all together and the biggest names in AI are turning into energy companies. They're paying for power plants, restarting reactors, and building their own supply. Because the chips they've already paid for need somewhere to run. So why not just make the chips use less power? If every chip did more work per watt, the same building could do more without a single new power plant. Nvidia says that's exactly what it's delivering. It claims Vera Rubin produces up to 10 times more AI output per megawatt than Blackwell. In plain terms, a hall that used to answer one AI request could answer 10 on the same electricity. That's Nvidia's own number, measured on a workload Nvidia chose. But the direction is real, and Nvidia says that across six generations of its chips, AI output per megawatt has improved about a million times over. Vera Rubin itself shows what happens to those gains. It's far more efficient than Blackwell, and each of its racks still draws more power than a Blackwell rack, because Nvidia spends the savings on packing more work into every cabinet. The whole industry does the same thing, which is why the power bill keeps climbing. When AI output gets cheaper to produce, companies don't bank the savings and switch off the lights in the hall. They run bigger models, longer conversations, and more AI agents working around the clock. Britain went through the same thing with coal in the 1800s, when better steam engines made coal power cheaper, and the country ended up burning more of it. The International Energy Agency has put a figure on where that leads. By 2030, it expects global data center electricity use to more than double, with AI as the biggest driver. That would be around 945 terawatt hours a year, more than all of Japan uses today. That's the hardest obstacle of all, because more money doesn't make any of it arrive sooner. Nvidia brings out a new rack generation roughly every year, while every fix in this story, the transformers, the turbines, the reactors, takes years. Hyperion's newest power plants are still waiting on regulators, and the Texas audit is still working through its list. The neighbors are pushing back too. In Temple, a local group is formed to fight more data center construction, worried about the heat these sites throw off. In Virginia, Texas, and Louisiana, regulators are now arguing over who should pay for the new power lines and plants: the tech companies or the families living nearby. Every one of those fights adds months to projects that were already running late. Temple is running now, liquid cooled and full of AI hardware, on a design that didn't exist when its first walls went up. But Kyber is coming, and every hall built around today's racks will face the same question Temple did. For now, the physical world is a harder limit on AI than the chips are, because Nvidia can design a new rack faster than anyone can build the hall, the substation, and the power plant to plug it into. The leaders are the companies that bring their own power. XAI moves fastest, Meta has made the biggest bets, and Microsoft has warned its chips could end up waiting on buildings. What could change that is more transformer and turbine factories, and chips that cut the power each job needs. The one thing to watch is 2017 when Nvidia plans to ship the first Kyber racks, and we find out who has a hall ready to switch them on. If the power coming in can't grow fast enough, the savings have to come from inside the rack, where GPUs burn energy passing data over copper. Chip makers from TSMC to Nvidia are racing to swap that copper for light using tiny lasers beside the chip. Watch this video next. It's on your screen right now.