This Breakthrough Could Make Data Centers 1,000x Smaller

[00:00:00] Speaker 1: For more than 50 years the semiconductor industry had a simple answer to almost every problem: make transistors smaller. And it worked incredibly well. But after attending several major technology conferences this year, one thing became very clear: that roadmap is over. And what's replacing it is far more interesting: a completely new way of building computers. One that could eventually shrink an AI data center spanning more than 100 football fields into something closer to the size of a refrigerator. No, that sounds completely insane. But after spending more than a decade building advanced chips, this is one of the most interesting solutions I've seen in years. Because it doesn't try to build a better transistor. It tries to change the physics of computing itself. And if you're subscribed to the channel, you're awesome and you're gonna enjoy this one. Right now, the largest AI companies are building data centers so massive that they require dedicated nuclear power plants just to run them. Many people think that AI energy crisis is caused by too much computing. But modern chips are actually extra-ordinary good at it. The real problem is moving information. Because right now, as AI shifts from chatbots to agentic AI, which are autonomous systems, this problem gets dramatically worse. Because instead of generating a single answer, AI agents increasingly search, reason, call tools, access memory, verify results, and coordinate with other models. In other words, every step requires data to move. And eventually, moving information starts costing more than computing itself because physics gets in the way. Inside a processor, billions of electrical signals race through tiny metal wires. And as electrons move, they interact with atoms in the material. We call this resistance. You can think of it as friction for electricity. And as any other friction, this wastes a lot of energy. That energy doesn't disappear. It turns into heat. Tiny amounts at first. But when trillions upon trillions of signals are moving every second across an AI data center, those energy losses become enormous. That means that a surprisingly large amount of modern computing is actually a battle against heat. For 50 years, we made computers faster by shrinking transistors. But the next 50 may be about something else entirely. Like making information travel less. Or even better, removing the costs of moving information altogether. And this idea leads directly to superconductors. Now, when you hear the word "superconductors", you probably imagine some incredibly complicated physics experiment. Something like LK99 and floating magnets. Or a setup resembling a quantum computer, which is so sensitive that any tiny disturbance from environment just ruined the computation. But the basic idea behind superconducting computing is very intuitive. Remember the problem of moving data we just discussed. It turns out certain materials, when cooled below certain critical temperature, enter a completely different state of matter. A state called superconductivity. And in this state, electricity can flow with virtually zero resistance. Which is remarkable, because you've just removed the biggest energy cost in modern computing. But that's not the clever part. The clever part is that they got rid of the transistor. Instead, superconducting computers use a different device, something called a Josephson junction. And despite the intimidating name, the idea itself is actually quite elegant. It's basically a sandwich where two superconductors separate it by a thin insulating layer. And when it switches, the junction emits an incredibly small and precise pulse of magnetic flux. Researchers call it a single flux quantum. And you don't need to remember that name. What is important is that this tiny pulse becomes the basic unit of information inside the computer. In a conventional processor, information is represented by electrical signals switching on and off. But every time they travel through the chip, some of their energy is lost as heat. And as a signal moves through the wires, we also lose some of the information. That is one of the fundamental reasons why modern data centers consume so much power. While superconducting circuits work very differently. Instead of moving relatively large electrical signals around the chip, they use tiny quantized pulses. And because the surrounding wires are superconducting, almost no energy is lost along the way. Which is a pretty remarkable idea. And the numbers start to sound ridiculous. A single switching consumes only around 1 mV compared to roughly 500 mV in a modern transistor. And that's about 500 times less voltage and potentially tens of thousands of times less switching energy. But the energy is not the single advantage. Another one is that these pulses are extremely short. Roughly one picosecond in duration. A thousand times shorter than a nanosecond. Which means the logic can operate at frequencies conventional processors will never reach. Modern CPUs typically run around 3 to 5 GHz. While these superconducting processors have already demonstrated operation beyond 20 GHz. And some circuits have exceeded 100 GHz. Because many of the limits that slow down conventional chips simply start disappearing. Now at this point you might be thinking. Cryogenic temperatures, superconducting circuits, all of that sounds very familiar. Like quantum computing. But here is the important distinction. The devices themselves rely on quantum mechanical effects. But this is not a quantum processor. Because it performs classical binary computation. There is no quantum superposition involved. No entanglement. No exotic quantum algorithms. And honestly, that can be one of its biggest strengths. Because the problem of quantum computers isn't building the hardware. It's building everything required on top of it. The whole stack. Including software and the algorithms. Quantum computers aren't faster versions of modern computers. They are an entirely different computing paradigm. While the superconducting circuits still perform classical computation, just running on a fundamentally different hardware. And this means the industry doesn't need to rewrite the decades of software just to use them. And that makes the path from lab to real product much more realistic. But if superconducting computing is so compelling, why it isn't everywhere? And this is where a story gets difficult. And the answer has very little to do with physics. For more than a century, researchers have known superconductivity was real. The challenge was turning it into something manufacturable. The devices were difficult to build. And very difficult to scale. And nearly impossible to justify economically. So despite of decades of promising research, superconducting computing remained trapped inside research labs. Until iMac decided to take another look. iMac is a research lab based in Belgium. And if TSMC and Intel are where future chips are manufactured, iMac is often where future chips are invented. It's one of the few places in the world where the semiconductor industry comes together and experiments with technologies that may not reach commercial products for another decade or so. And recently, iMac decided to revisit this one of the oldest computing dreams. Superconductivity. Not because the physics suddenly changed. Because for the first time, superconducting computing start to look more commercially realistic. The question is if the remaining challenges are worth it. And we will deep dive into that in a moment. But before that, as someone who travels a lot and runs multiple teams across airports, calls and meetings, I really appreciate good communication quality. I've been testing the new Soundcore Liberty 5 Pro series and I'm impressed. These earbuds are built around Anker's custom AI audio processor using a compute-in-memory architecture that fuses processing and memory together to highly improve the performance. With 10 sensors constantly monitoring your environment, the earbuds can isolate your voice in real time, even in chaotic surroundings. And for how I work, taking calls between flights or jumping into meetings from wherever I can be, that availability matters a lot. They even set a Guinness World Record for call clarity in noisy environments. Then there is an active noise cancellation. It continuously analyzes the environment around you and generates the opposing signal needed to cancel the noise dynamically. The Pro Max version adds a touchscreen case and AI note-taking features. And you can control the playback or switch ANC modes directly from the case. The feature I ended up using the most is AI note-taker. I just double-tap the case, it starts recording, transcribes the conversation, generates a summary and even pulls out action items automatically. And that genuinely simplifies my day. Because instead of trying to remember what was agreed to in the meeting, all the notes are already organized for me. If your calendar looks anything like mine and you value communication quality, make sure to check them out in the description box below. Now, I make sure that many of the problems that held the technology back may actually be solvable. Their approach is built around the material called niobium-titanium-nitrite. And unlike many other superconducting technologies, it can be integrated into existing semiconductor manufacturing recipe. These devices can be fabricated on standard 300-millimeter wafers. That may sound like a small detail, but it isn't. Because it means this technology finally has a plausible path to mass production. But the really important innovation is hidden inside the sandwich, inside the Josephson junction itself. In the tiny layer separating the two superconductors. For decades, that layer was one of the biggest obstacles to scaling this technology. iMac replaced the traditional aluminum oxide barrier with amorphous silicon. And that change may sound minor, but it makes the device dramatically easier to manufacture at the densities required for real computing systems. Except for one problem, which is impossible to ignore. This is actually the reason why you don't have one of the superconducting circuits in your laptop today. Superconductivity only works when things get extremely cold. We are talking about roughly 4 Kelvin. That's minus 269 degrees Celsius. Just a few degrees above absolute zero. And that means you cannot simply put one of these chips into a normal server rack. Instead, the processor sits inside the specialized cryostats. You can think of a cryostat as an ultra-advanced scientific refrigerator. Its job is to keep the entire computing system just a few degrees above the coldest temperature physically possible. And that's usually the first criticism. Does any of that make any sense? After all, cooling something to 4 Kelvin sounds incredibly expensive. Wouldn't the cooling system consume more energy than you save? Surprisingly, based on the iMac research, the answer is no. For small systems like your laptop, this doesn't make any sense. But the larger the system becomes, the more advantages it you get. In fact, according to iMac's analysis, there is an inflection point. Below a certain scale, cooling a superconducting computer costs more energy than it saves. So it makes no sense. But once you reach the scale of today's AI infrastructure, the math flips. The energy savings become way larger than the cooling penalty. But there is another advantage, which is even more important. Density. And to understand why, it helps to imagine what one of these future systems may look like. At the center sits a superconducting processing unit. Think of it like the superconducting equivalent of today's GPU. But unlike a conventional chip, much of this hardware lives inside a bath of liquid helium at around 4K. Then, through a thermal bridge, the system connects to a warmer region, running at around 77K. Still incredibly cold. But warm enough for conventional silicon DRAM memory to operate much more efficiently. Beyond that sits the normal world. In a sense, it's a computer system built across three different climates. And this is where it starts to get really interesting. For decades, semiconductor industry increased performance by shrinking the transistor. But because of the physics wall, now it gets harder and harder. So what's happening? We are trying to move. We are trying to build transistors and circuits and chips in the third dimension. Stacking memory on top of processors. Stacking chiplets. And eventually stacking entire systems. But the problem remains. It's heat. Modern processors already struggle to cool a single layer of silicon. Stack enough layers of logic on top of each other. And the chip eventually cooks itself. That's why today we are moving towards stacking memory on top of logic. Not logic on top of logic. Because this will simply burn the whole thing. And when we are able to stack memory on top of logic, we are able to shorten the distance. And increase the bandwidth without the killing the thermal budget. But superconducting systems completely change that equation. Because they generate so little heat, that logic chips can potentially be stacked directly on top of the other logic chips. Layer after layer of computation packed into a dense three-dimensional structure. That means shorter distance for data to travel and higher bandwidth. In fact, IOMAC modeled such a future system which would be built from 100 superconducting circuit boards. And it would fit into the size of a shoebox. And deliver over 20 exaflops of compute. That's roughly 20 times more compute than today's largest supercomputer. And it would consume only about 500 kilowatts of power. Not hundreds of megawatts like modern data centers. And if those projections hold, we are talking about at least 100 times improvement in energy efficiency. And the huge idea behind it is not to build a faster computer. But to build a denser computer, which can deliver drastically more compute from the same amount of power and from a way smaller footprint. Because in the AI era, power and space have become some of the most valuable resources in computing. For a laptop, cryogenic cooling makes no sense. But for an AI data center consuming gigawatts of power, the economics looks very different. And that's exactly why the AI industry is paying attention. And interestingly, this is not the only attempt to solve this problem. Recently Huawei proposed so-called logic folding. We found the breakthrough. We named it the logical folding. Which is extremely ambitious claims. And I'm working on an episode about this. If you don't want to miss it, remember to subscribe to the channel. Now, here is something interesting. We don't have a factory for superconducting logic. At least not yet. But the industry is already investing billions into something closely related. Quantum computing. Right now, IBM is building a massive new manufacturing facility near New York fully dedicated to quantum technology. At the first glance, that may sound unrelated. Quantum computers and superconducting logic are two different things. But what's interesting, underneath, they share the same engineering challenges. Both of them rely on superconducting materials. Both operate at cryogenic temperatures. And both require advanced packaging and specialized manufacturing processes. as well as electronics that can function near absolute zero. In other words, while IBM is building factory for quantum circuits, they are, by the way, solving many technology challenges needed to commercialize superconducting circuits. Because for decades, superconducting electronics had a chicken and the egg problem. Nobody wanted to invest in manufacturing because there were no products. And there were no products because there was no manufacturing ecosystem. Now, quantum computing is helping break that cycle. And for the first time, there is a serious industrial effort to build computers that operate just a few degrees above absolute zero. And that's why I find this technology so fascinating. But AI is exposing a new bottleneck. Distance, moving information has become expensive. Heat and power have become expensive. And superconducting logic attacks all three at once. Not by building a battery device, but by rethinking how information moves through a computer and optimizing everything around it. Now, to be clear, I don't think superconducting logic will replace silicon. Your phone and your laptop will remain CMOS-based for a very long time. But for frontier AI data centers and future quantum systems, the equation starts to look very different. Because once moving information becomes dramatically cheaper, something interesting happens. Compute is no longer need to be concentrated inside ever larger facilities. But it can become denser, more efficient, and way closer to where it needs to be. Closer to factories. To the available power grids. Much closer to the real world. And that may be the biggest shift of all. If you enjoyed this episode, you will definitely love this one, where I break down the most fascinating microchip factory being built right now in Texas. Or this one where I explain how particle accelerators are saving Moore's law. Love you guys, and I will see you there. Ciao.