The AI data center oversupply crisis is coming — Ed Zitron

[00:00:00] Speaker 1: I think we're going to see a supply glut. We spent half a trillion dollars to get one more sales force or two more sales forces run by some of the dampest, most annoying people alive. Jesus Christ. This is a sign they don't need the compute. And it's also a sign that perhaps they overbuilt. And if they overbuilt, I think it's fair to ask whether everyone else did too. And also, this is the dot-com era all over again. If Meta sells compute to Anthropic, it's game over. We are at the end. We're in this weird situation where everyone wants them to reduce their costs, but if they reduce their costs, they reduce the need for AI compute, which means we've overbuilt the supply. But if they don't reduce their costs and they need all that compute supply, they need endless venture capital and debt, which will run out. [00:00:50] Speaker 2: Hi, and welcome to The Tech Report. I'm Isaac, and joining me today is writer of Where's Your Ed At and the host of the Better Offline podcast, Ed Zittrain. Thanks for coming back on. Thanks for having me. Until now, the general assumption has been that the demand for memory in semiconductors far exceeds supply. But this might no longer be the case, as questions of an oversupply of compute has kind of panicked investors after the first hyperscaler has started selling its surplus. Now, this is Meta we're talking about. So the obvious question first is, do they just not need the capacity because they're not popular enough to use it all? [00:01:28] Speaker 1: Well, that's the thing with Meta. I don't know if anyone else has been counting, but I'm now at five different reorganizations of their AI department. They've come up with three or four different random... They had Llama, their open source one, which people liked, but it was open source and thus no one would pay for it. And so they built, I think they spent over $100 billion in CapEx. Then they spent $14 billion in scale AI. Then they built the Muse Spark AI model, which was immediately middle tier. So they spent all that money to not be the best. And now they are... And this is, to be clear, this is an internal discussion at this point. They're talking about leasing out their compute to someone else. This is a sign they don't need the compute. And it's also a sign that perhaps they overbuilt. And if they overbuilt, I think it's fair to ask whether everyone else did too. Now, Bulls and the recently concussed will suggest that, oh, well, this is actually a super smart play from Meta because it's a way of monetizing their CapEx when they haven't been. What it actually is, is a tacit admission that they don't need it. That they actually have no ideas, that they never have had a real plan for this. And even, I think, less than a month ago at the annual shareholder meeting, Mark Zuckerberg said, we think we have a use for the compute, but if we don't, we'll rent it out. So it's just, this is what a company does when they're out of ideas, just like Meta always has been. It also begs the question as to who's next, and also when Meta will cut CapEx. Because at this point, if they keep spending on CapEx after saying this, what exactly is the plan? Are they going to become CoreWeave 2? Are they just going to be the largest AI compute company? I don't really know what the thing is. The other problem Meta has is that the majority of their compute is an older generation's GPUs, H100s and H200s. So it's just, they're going to become the largest, oldest GPU provider. It's a mess. And I think the answer is, well, there really never was a use for this compute in the beginning. And I think that we're going to, the only reason that Amazon, Google, and Microsoft aren't doing the same thing aggressively, like trying to find places for it, is Anthropic and OpenAI take up the space. Which is why my prediction is, is that if Meta sells compute to Anthropic, it's game over. We are at the end. Because that will just be a sign that the only people that will buy compute at scale are two bulbous, unprofitable fail sons. [00:03:50] Speaker 2: I mean, to your point, Meta is still on the books to spend, I think, between 125, 145 billion this year. Yeah. And yeah, to your point, why if they're selling surplus? [00:04:02] Speaker 1: Well, there's another question with that as well. They are, they have a deal with Nebius for, I think, over $17 billion. They have a deal with CoreWeave for over $30 billion. Are they going to resell that compute? What's the plan there? Because especially in, I actually think in both of those cases, both of those companies took those contracts from Meta and used them to raise money to build the data centers for Meta that Meta might not need anymore. Kind of a fair question to ask what's going to happen there. And what about Hyperion? Their $30 billion data center in Louisiana, it's apparently meant to be the size of Manhattan. What are you going to do with that, Marky Mark? What's the plan? Because it's very clear that nothing is going on there. Alexander Wang is the head of whatever division runs AI for Meta anymore. It's not really clear. He went on Twitter and he was, because Mark Zuckerberg made the comments saying that the progress of agents has not been as fast as expected. And Alexander Wang went out and he said, well, actually, Mark Zuckerberg was talking about the entire industry, not just Meta, which I love, by the way, because this already soured the mood quite a lot, this discussion. And Alexander was like, yeah, just to be clear, we don't, we mean everyone. We don't just mean us. Just in case you were worried that it was just a me thing, it's a all of you thing too. And so the bulls are furious, but they're all coming up with little ideas. I truly don't know why they're still spending CapEx. I don't know what the, like, what does, I think, but I will be honest, I actually think this is a fair question for every hyperscaler. What is it you're getting by building more outside of just doing, like running high-end welfare checks for Anthropic and OpenAI? I'm not really sure what more CapEx is doing, because we're what? We're over a trillion dollars now. Where are the outcomes? Like, what's, what has this done for Microsoft, Google, and Amazon outside of kind of pumping their own money back into their revenue? It's definitely not done anything for Meta, other than that story from Reuters about an old man being led to his death by a Kylie Jenner bot. That's a real story. It's an insane, crazy story. It's insane that that happened. And I think, I think that Meta is in the running to be the first to pull CapEx back. I've seen semi-analysis said, oh, actually, they're going to spend more. And it's like, why? Why? What is, what is it with this industry? What is it with the AI industry that every time reality dawns, they're like, no, we must be stupider now. We must, we must escalate even further. [00:06:36] Speaker 2: You say Meta might be the first one to pull CapEx. We've also seen, if you count SpaceX, maybe not as a hyperscaler, but they have also sold surplus to Anthropic. Who do you think might be the next person to start selling surplus? And then where does that leave the hundreds of billions of dollars of AI infrastructure that is still slated for construction? [00:07:00] Speaker 1: I mean, it's Meta. I think Meta, if Meta ends up selling to Anthropic or OpenAI, we really are at the end. Because remember, I think that they have like one to two gigawatts of capacity. If they end up selling it to OpenAI and Anthropic, that's them saying, we don't have another big customer. Or there is not enough diverse demand to sell this to someone else. Microsoft, Google, and Amazon, they don't have surplus capacity because Anthropic and OpenAI take it all up. We have already seen CoreWeave, their biggest customers are Microsoft for OpenAI, OpenAI for OpenAI, Google for OpenAI, Anthropic, and Meta. Oh, and also NVIDIA for themselves. And then there's Nebius, Microsoft for OpenAI, Meta. I mean, it's the same thing every time. Cypher Mining, Anthropic, Iron, Anthrop, sorry. And Iron might have been Anthropic as well, but that's the thing. It's always the same names because there's not really the demand outside of it. At this point, I think there's just a compelling question to us, which is why are we doing this? What is this for? Is it just, is this industry existing to fuel Anthropic and OpenAI? Because that's a problem as both of them are unprofitable. It's just, it's feeding venture capital into hyperscalers or into hyperscalers so they can feed it into NVIDIA and the Taiwanese companies that build their servers. Like what is, and now SoftBank is apparently planning to build their own capacity, which is equally bonkers. It's just, I think we're going to see a supply glut. I think we're going to, if Meta chooses to do this, if Meta chooses to rent to OpenAI and Anthropic, by the way, I think that that's to avoid a supply glut. That's to avoid the industry really reckoning with the fact there's too much supply. But I think that, I don't know who else would have spare capacity. Elon Musk is already selling all of his. He already sold it to Anthropic and to Google. Google, I think 900 million or something a year. It's unclear. And also when you line up the amount of GPUs that both Anthropic and Google are renting, they kind of like, there's not enough in the, they're using too many GPUs in both deals for both of them at the same time. I mean, is this industry capable of doing anything that isn't circular financing? Is this industry, Meta is actually going to be the test of this. Because if Meta comes forward and they say, we're going to do a standard inference operation where we're going to offer it to whoever wants to run open source models or rent R models, we're going to do piecemeal contracts, fine. Then that, that's like, that's a respectable ish. I don't think they'll have the demand, but it will be, okay, we're going to have a real go at this. If they just sell it straight to OpenAI and Anthropic, that's because they know. It's because they know there's not actually going to be outside demand. It's just kind of a mess. I think we're really going to look back on this era and say, why did we do any of this? [00:09:48] Speaker 2: Even if we assume that all of the companies at play remain afloat somehow throughout all of this, how much oversupply do you think there is going to be at the end of this period? [00:10:00] Speaker 1: I mean, it remains to be seen how long it takes them to cut back capex. Because I think, let's say 10 gigawatts comes online in the next two years. I don't think it'll even be that much. Anthropic and OpenAI, I think, I estimate take about three and a half gigawatts total, which is the majority of available compute. And the rest of it goes to the internal services, Google, Microsoft, Amazon, much more Google and Microsoft. I think that there is probably about net net outside of Anthropic and OpenAI, maybe 500 megawatts to a gigawatt of capacity demand. I think that that's it. I don't think there's very much more. I've looked around, I've looked at Lightning, Base 10, all of these companies, and I've tried to work out how much capacity they actually have. And it's not that much. And even then, and the amounts of revenue they get are vanishingly small. Like, I think Lightning had 500 million annualized revenue, which is got like 30, 40 million dollars a month. Not that, like, that's a lot of money to you and me. But for the amount of money, the capacity costs kind of not that very much at all. So I think, as I said a few months ago, the thing that's keeping there some sort of supply constraint is the fact that Anthropic and OpenAI take up everything, and indeed, the data centers have taken a while to come online. I think that ultimately what will bring this to a head will be someone dumping a bunch of capacity online, like Meta, if they actually just dump it onto the market. Or alternatively, stuff will come online, and at that point, people will say, oh, well, now I can find a GPU for cheapest chips. I can find one anywhere. It doesn't cost me anything. It will be, what's going to be interesting is, so there's two ways you buy GPU compute. You either buy an allocation for a year, or two years, or what have you. You buy, like, a few hundred, a few thousand, or you buy spot prices. Whenever you see people talking about the price of GPUs going up or down, it's the spot price. The spot price is not indicative of the general availability. It's literally just what's available at that time based on who's got anything free. If spot prices start to crash, however, that's a sign that there's just a ton of capacity available. The question is, when do these bloody data centers get built? Because we truly don't know at this time. It's really difficult to actually get an update on any data center project. [00:12:20] Speaker 2: I know this is something we've covered quite extensively, but what happens if a hyperscaler does pull back on the aggressive CapEx spending on AI, especially given sort of how much debt there is involved in it? [00:12:31] Speaker 1: So I saw a fellow called Rich Garrodowski, I think it was a Goldman Delta analyst, who said that the first hyperscaler to pull back on CapEx will be rewarded. Once that happens, all the others will choose to do it so they get rewarded. They have the brains of dogs, the hyperscalers. They may have so much money, but they have very kind of blunt thinking of, oh, market happy, oh, market sad. And I thought Meta was going to be the last man standing. I actually think they could be the first. All it's going to take is one of them to do it for the others to cut back. And when they do so, it's not going to be, we're cutting CapEx because AI isn't working. They're going to say, era of efficiency. We're doing efficient AI. This is an attempt to make AI efficient. We're going to make sure our AI stuff works. We're being moderate with our costs to benefit shareholders. They'll bump share buybacks. Like they'll do whatever they can to make it seem like a good thing. But I think the market will see through it because they're already, even with this meta capacity story, Nebius, Iron, Core Wave, all of them dumped. I mean, even the applied digital, all of the exterior AI compute companies dumped on the markets. So I think what is going to happen is one will do it and the rest will follow and the markets will reward them. But the thing I've been saying for years is, okay, when AI is done, what's next? And there really isn't anything. So I think that, I think it's going to be a case of meta doing it, then others following. But the question is who? It could be Microsoft, but Microsoft, Satya Nadella, the CEO, recently posted a thing about spending a billion dollars on forward deployed engineers. He has full AI psychosis. Could be Google, could be Amazon. I think Amazon is the candidate to at least do a little bit of cutting because they were planning $200 billion this year. I think it really will be, it's going to be interesting to watch because they're an industry of cowards. They're cowards. They don't have any ideas. They just, they're doing this because they have no other hyper growth ideas. So they just, they could stop it for equally thin reasons, for equally flimsy reasons after being so excited about AI. And also, if they, if Microsoft, Google, and Amazon cut CapEx, that will be very interesting as far as OpenAI and Anthropic go because as I've said before, those two companies have never built any of their own infrastructure. None of, neither of them have. I estimate that it's cost about a quarter of a trillion dollars to build the hyperscaler Cap, in hyperscaler CapEx even, to build our OpenAI and Anthropics, just very basic infrastructure in the Altman versus Musk trial. A Microsoft executive said they'd spent a hundred billion dollars or more on their partnership. They've only invested 13 billion. So I think it's safe to say, and that was, well, earlier in the year, I think it's safe to say they've spent 80, 90 billion dollars just on the infrastructure for OpenAI. And that's before you consider that Amazon's built up stuff for OpenAI. It's probably 200, 250 billion they've spent on this. So OpenAI and Anthropic have never had to invest in infrastructure. Now that they'd have to, I don't think they can afford to. I don't know what they do. It's going to be a mess. And I think the sooner it happens, the better, because the market has become so overheated over this stuff that I'm just not sure what to, I'm not sure even what the purpose of investing further is. This isn't even a bearish take. This is just, what are you doing? Like, why are we still doing this? Are these companies ever going to leave Pinocchio status? [00:16:13] Speaker 2: So on a bit of a side note, the information reported that an OpenAI engineer had found a way to half inference costs. And I just wanted to quickly ask what you know about that, but then also the question it raises. Because if a major efficiency gain does happen, it may alleviate some of the pressures around sort of AI being subsidized, even though half would still be not profitable or anywhere near. Yeah, it raises questions around, like, does it then deepen the oversupply problem that we're talking about already? [00:16:47] Speaker 1: Well, first of all, the information, I paid for the information for years. I really like them. This story was weird. It's very weird because I'm looking at it right now. It's, OpenAI engineers earlier this month told some colleagues that they figured out a way to more than half the cost of inference. But then it goes on to say, when the engineers applied the new techniques to power ChatGPT for visitors who didn't have a free or paid account, it's just like, no, they haven't halved the cost of inference. They have found a technique to halve it in one situation, maybe. If OpenAI had actually found this, they would do a, they'd do a blog. Sam Altman would make an annoying lowercase tweet about it. They, when they talked about a jalapeno chip with Broadcom that may or may not reduce costs, they would play in saxophone out in Times Square. They, not literally, they were excited and they were talking about it. They would go on CNBC and say this. It wouldn't be what appears to, I'm guessing, be a Slack conversation which someone got sent. What I think this is is likely someone on Slack in OpenAI went, hey, Amaya found a way to half it. I tried this one thing. What do you think? And that blew up. Even the AI boosters were like, this is a nothing burger. When they are saying that, you know it's nothing. As is the thing. I don't think that OpenAI magically found a way to half inference. I don't think that that happened. I just, I really don't because they would make a much bigger deal. It would be a big deal. But you're right. If they actually did this, it would mean they needed less compute, which would lower their demands for compute. This is the thing now. We're in this weird situation where everyone wants them to reduce their costs, but if they reduce their costs, they reduce the need for AI compute, which means we've overbuilt the supply. But if they don't reduce their costs and they need all that compute supply, they need endless venture capital and debt, which will run out. And it appears that they're just going to linearly need more compute for the rest of time. Great. We've just got two unsustainable tracks going on. And now this growth of open source models that's scary to them. And we can talk about Sonnet 5 if you want, but there is just a certain degree here of no one has a solution to any of these problems. We either have so much demand from two companies that means that we need to funnel them money forever, or these companies will find efficiency gains, which will mean they don't need as much compute, which will mean that we've built way too much compute because the demand doesn't exist outside of them. Both of these situations are bad. Both of these situations end in tears. There is, because if they reduce the cost of inference by half, they would still have to train, which would still cost them way more than the money they make. Also, if they reduce the cost of free users by half, they still have the expensive users. I think the, just thinking about what it could be, they could be serving free users' inferior models, they could be serving them slower, like just the actual compute, they could batch the requests or something so they come out slower. I'm sure there's a way that they've made it cheaper for free users by making it worse, but I severely doubt they've halved all inference costs. If they did, I'd love to hear, but again, it creates the problem that you discussed, which is, yeah, if they halve their inference costs, they halve the amount of compute they need, which would be bad for everyone. I don't think that that's what happened though. [00:20:16] Speaker 2: I want to talk about NVIDIA for a second because they launched a new partnership program where they offer to lease or buy back GPUs from cloud providers in exchange for a percentage of their profits. But obviously that is, to me anyway, that sounds like a terrible deal for the cloud providers, but leaving that aside, it seems like the whole purpose of that partnership program is to reassure customers who are worried about the risk of a massive oversupply problem. No? What do you think? [00:20:46] Speaker 1: I think it's something more evil than that. I think that, by the way, at the end of this era, we need to make circular finance illegal. I just, I think we need to end this because what NVIDIA is doing is NVIDIA is saying, if you buy our GPUs, we will guarantee to rent them back. And he's specifically saying it to new Neo Cloud types. This should be illegal. I just really want to be clear. This is not real. And also, this is what you do when real demand doesn't exist. And the bulls, the mold poisoned, whomever, will say, well, actually, it's just NVIDIA giving reassurances to customers and the demand exists. They call Professor X about it. Oh, oh, I've worked this out. I've got the solution. No, this is an act of desperation by NVIDIA knowing that we don't have a functional SEC at the moment to say, hey, are you round-tripping? Round-tripping is illegal. You're not meant to do it. Are you round-tripping? But what they do is they do it at such small amounts it kind of goes, it's under the radar, even though it's reported in the information. NVIDIA is doing this because the compute demand doesn't exist. Like, it's that simple. It's not a complex point. People want to make it complex. People want to turn it into this whole thing, oh, it's just how supply works. It's like, no, this is what you do when you know no one's going to rent these damn things. And I also think it has something to do with debt because CoreWeave, Nebius, Iron, all the NeoClouds, the way that they raise money now is they take a contract from a hyperscaler. So Meta, let's say Meta, says we're going to pay Nebius 17 billion and what have you. They take that to the bank and they go, look, we have a big, sexy cloud provide. We're going to provide the compute to these people with guaranteed money. And then the bank goes, oh, well, they're going to have guaranteed money, so we'll give you the loan. NVIDIA has done this already with CoreWeave. They literally gave a contract. They agreed to buy back CoreWeave's supply so that they could raise money to buy more GPUs to put in the data center. Again, this should be illegal. This is really bad. This is the end run. The fact that we are still doing this, if this was happening two years ago, which it was, it's been happening since 2024, Project Osprey, it was called, with CoreWeave and NVIDIA, I would get it. I kind of got it in the early days, sure. I can see the argument. I don't love it. We're in the year of our Lord, 2026. It's been a while now. It's been a long while. We've had the, we've had generative AI for a minute, Jensen. You shouldn't have to keep paying your customers to pay you. And also, this is the .com era all over again. It's Lucent. It's Nortel. It's all the, it's all the great, we're doing .com again. It's, we're doing all, we're playing the hits because there is no there there. There is not the demand and the demand is not magically arriving. So these, at some point, this just, the bottom falls out of this. At some point, a bank is going to say, hey, Jensen, love you, baby. We love giving you money. However, I just don't believe that, like, I don't think that this is a stable relationship. Because remember, NVIDIA, incredibly wealthy, incredibly profitable company with a ton of money. They have $26 billion over the next five or six years in cloud compute commitments. As in, to rent, they are spending $26 billion to rent back their own GPUs. Again, sign that there's not real demand. They also have made, I think, $70 or $80 billion worth of impossible-to-cancel commitments to TSMC. NVIDIA, in the event that, I don't know what their cancellation terms are with their GPUs, but if their terms allow cancellations and people cancel, much like the dot-com bubble, then NVIDIA could not afford their bills and could not afford to pay these deals. There is a way this unravels even for a very wealthy company like NVIDIA. I'm not saying NVIDIA dies or anything, though they are setting themselves up to make that not impossible. It's very, very, very, very unlikely, but some of the upfront commitments they've made are genuinely bonkers. I think at some point the banks just get tired of issuing this debt because I think at some point, it's not the student loan business. It shouldn't be that every single loan is co-signed by someone's parents. You eventually want the debtor to be able to pay you. It's just a very, it's all very unstable and it drives me a little insane because you speak to leading journalists and analysts and such and they're still to this day going, AI, chips, it's all natural and good. We love it. If that was the case, why is everyone acting so weird? Why is everyone so shifty? Why is everyone like, yeah, you know, I'm going to buy your GPUs from you, great, but can you pay me to buy them from you? That's not a real industry. None of this is. It's kayfabe. It's everyone pretending this is real in the hopes that it becomes real. It's like a, like Valley Girl manifestation except on trillions of dollars at a time. So, completely threw me off [00:25:47] Speaker 2: with that. Yeah, sorry. So, yeah, if we do look at the memory markets, Samsung and SK Hynix announced last week that they're planning to spend over $500 billion on expanding their memory fabrication capacity, which, when you combine it with what we've been talking about, MESA and the general concern of oversupply, it goes a long way to explaining why we did see those chunks taken out of memory makers that you mentioned earlier. And do you think, maybe this is the gamer in me being a little too optimistic, do you think we could be going back to a time where memory is being sold? Maybe, maybe not at the, where it was before being basically sold at or below cost, but a lot cheaper, not 700 times where it was last year. [00:26:38] Speaker 1: So, the memory, the thing with memory is it's a boom and bust industry. Only two years ago, three years ago even, Micron and them were in real trouble because memory is classical memory. So, DDR and all that, the stuff in your computer is not super high margin. High bandwidth RAM so the kind that's actually on the GPUs themselves, extremely high margin, but they take up more space on the fabs where you actually build the RAM. I'm simplifying, someone in the comments is going to say this is too simple, shut up. Anyway, but basically, so their fabs are being taken up more by this very expensive RAM that's basically just going to NVIDIA or it's going to other GPUs and ASICs. Then, so, all of that's happening so that's just taking up more room, leaving only a little bit of space for the regular RAM which they can charge whatever they want for now because there's a supply shortage. It's going to take years to fix. I just, I hate to say it, but when you, when this era is done, hating the AI industry, you need to hate them way more than you already do in my opinion because it's not just all of the horrible stealing and the, all of the money they take and the lying and the misleading and the nonsensical economics. It's the fact that because of the demands of basically one industry and building data centers for basically two companies, it's going to crank up the cost of RAM until 2028 minimum and the problem is, is the, before this point, SK Hynix, Micron, they were saying we're not going to massively increase our capex. We're not going to overextend ourselves because memory is boom and bust. You get a big time, you fill your boots, you fill your coffers and then you say, okay, this is going to crash, no problem. Except now, SK Hynix is saying that they're going to remove the price caps on their long-term deals, meaning that they can just screw their partners all the live long day. I think you have to do a minimum agreement in three years now versus one year. That again, not that scary. What is scary is this 500 billion investment. The only way they can fund that is if there's continued business for their memory. If that, if they make commitments that they can't fulfill, they are going to be in financial trouble. Again, again, three years ago, all of them were in the same trouble because they massively oversupplied because there was the post-COVID thing where everyone wanted RAM and now no one wants RAM. This is going to be far more dangerous though because AI bubble bursts, the use of HBRAM is not, it's not going, we're not going to need all that VRAM anymore. We're not going to need quite as much, which means that they're just going to be sitting around with a bunch of RAM that has uses, there are other uses for iBandwidth RAM but nowhere near as much. So it's going to create this situation where they will have overextended themselves. NVIDIA will have overextended themselves. Microsoft, Google, Amazon overextended. The whole tech industry is going to be cash poor for a while. I think the only way they fix that is by charging way more. I actually don't know but what I do know is that these companies, these memory companies really think they're living high on the hog right now and there's one argument to be made that they got kind of screwed in previous eras but I think it's hard to, I'm paraphrasing Steve Burke from Gamers Nexus, it's hard to feel bad for a company like Micron when they're screwing gamers so badly and when they have 84 point something percent margins. Like they didn't have to do that. Like they could have 70% margins which would be still great for memory but they're like no, we have to and it's because everyone wants it so we're cranking it up and it's just, it is an unhealthy way to run a business and perhaps we need the memory companies to have a more consistent business line which would mean in aggregate the prices raise a bear but what they're doing now is nakedly evil and it is completely caused by the AI industry. It's caused by the demands of high bandwidth RAM and when the crash comes it's going to be so much just like I've been saying it's going to be so much worse the longer this goes on because now the cost of pretty much every consumer electronic is going up. Even the cost of used consumer electronics. Go on eBay right now and look for a laptop and see how many of them are sold without RAM. It's truly disgusting actually and it's in storage now with a solid state storage and Nance storage and all that. It's just, it's truly disgraceful and I don't, I don't know how they're going to spend $500 billion. I hope they don't begin construction too much because they're not going to have the money to pay for it. Now to be clear nothing's going to happen to Micron or SK Hynix or Samsung especially the last two. They're, I think they're shyballs which are basically like quasi-nationalized Korean companies. I think they will be fine. Nothing will happen to them especially Samsung. Micron will be fine because the industry needs them but it's, it's frustrating because as always when this bursts the financial crisis that will follow will hit regular people. When the memory bubble bursts the cost won't come down it will just be rough for everyone involved and ultimately the consumer will suffer. Again, the AI industry I don't even think gets a hard enough time because they cause this. Every single person who inflated this bubble is to blame for the cost of RAM, for the cost of storage, for the cost of everything going up. The inflation caused by the AI industry is their fault and they should suffer for it. They won't. The rich people will be fine at the end of this but we need to know who the enemy is and what they have done to us. [00:32:06] Speaker 2: So just to kind of bring this full circle in your free newsletter you kind of put it very succinctly that this sort of trillion dollar of hyperscale CapEx is essentially just feeding a massive semiconductor boom on the hopes that LLMs turn into something that they're not. Something completely different. Yeah. And currently the rate at which LLMs are improving is starting to plateau and open source is starting to kind of catch up and we have talked about this in loose terms before but how much of a leap is needed to make the build out make more sense even if we kind of do put aside this [00:32:41] Speaker 1: excess [00:32:42] Speaker 2: of compute. [00:32:44] Speaker 1: They would have to actually do the things that they said LLMs would do. So culturally when we think of AI we think of set it and forget it and we think of autonomous intelligence you know AI. It's meant to be something where it just works. You don't think about it. I don't even think you'd prompt it. It would just do all this stuff in the background magically and it would just work. You wouldn't have to worry about hallucinations so you just have to solve the mathematically certain hallucination problem. By the way when I say it's mathematically certain that's from OpenAI's own research. So yeah you just have to fix that you know just maths just fix maths just fix maths first. It would have to be the computer from Star Trek. It would have to just be like I want this and it would pop out and it'd be great. I want to clone Slack and it would have a Slack clone and it would work completely and it would be provisioned and it would be secure and it would be online it would be on AWS you would have all the CDN stuff set up. All of that would just have to happen. It would basically have to be a thing it's not. It would cause an immediate job apocalypse. This is like but I am basically talking in fantasy terms. I may as well be talking about wyverns and dragons and such. Wizards. Because that is how realistic this is. And also I don't know that sounds like a completely different product to me which suggests all the capex so far has been a waste. So it would have to be a product so good that it did all the things they promised and it would have to be so good and so revenue generating that it would make up for the fact that they basically wasted a trillion dollars getting there because it is a waste. If they somehow work this out which I really don't believe they will it will be because of something they've done from today onwards. It's not going to be anything to do with what they've done before. Large language models are not going to do that. And you can oink and squawk in the comments all you want about oh well I've used it for coding. It's not good enough. It's just not good. It's not good enough. Doesn't make any of this worthwhile. Sorry. For it to be worth it it would have to be a product where they could get basically a hundred bucks a month from every human on earth and they would have to pay. It would have to and to be clear that would mean a product more sticky and relevant than Spotify or Netflix. like just they make 30-40 billion dollars a year I think Netflix it would have to be like 10x that between two companies and it would have to be consistent revenue because right now I think we're going to see we haven't had a report on new revenues from Anthropic in a minute. We haven't had a new ARR. Perhaps they'll have one more bump and then I don't know but everyone's cutting back on that Tesla is limiting people to 200 bucks a month sorry 200 bucks a week of AR use. Everyone's cutting back. UBS had a study saying 60% of enterprises are doing some kind of token minimizing. This is a sign that their revenues are going to go down because they moved everyone to token based billing so they would have to get something that was real consistent recurring revenue which would mean they would have to have lowered costs so much that the subscription model works because there's no there is no metered business on earth other than gasoline and electricity that actually works like so they have to lower the cost to effectively nothing and make the end product so impossible to avoid and so useful that it would mean that everyone had to use it and I'm just what we're describing is the antithesis of LLMs LLMs are like oh the fuel it's insanely expensive the output's extremely unreliable when we say AI we think of Commander Data from Star Trek we think of this ultra intelligent completely 100% reliable cold rational completely functional thing and with LLMs it's like alright can you get it to do this kind of but you need to prompt it right and if you use the wrong harness it's not going to work and yeah you're going to have to make sure you have to check it doesn't do this because if it does if it breaks this well then everything breaks ideally when you do research using LLMs you have to know the subject matter completely so you can tell if it got something wrong at which point it's like if you needed someone to be confidently wrong you could just talk to anyone online literally anyone you find someone in two seconds and be like yeah yes yeah yeah I think America was founded in 2001 I don't know mate I didn't really look recently that is an LLM to me no matter how much it gets right it will always get something wrong so yeah the answer is for this to work out they need something else and that's something else needs to be so overwhelmingly good that it needs to completely wipe out the past it needs to just be like they need to because it's no longer sufficient for this to just become a regular business let's say best case scenario open AI and Anthropic fudge together some profitability and they became Salesforce sized which is 40 50 60 billion dollars of annual revenue and with like I don't know 50% gross margins what their combined infrastructure cost I estimate along with their funding is about 540 billion dollars we spent half a trillion dollars to get one more sales force or two more sales forces run by some of the dampest most annoying people alive Jesus Christ ugh but that's the thing in that let's take that scenario for a ride for a second great Microsoft Google Amazon wow so they've made they have two very large new clients and businesses that kind of work but LLM still mess up in the way they do and people use it for coding I guess it's still unexceptional and it's still not enough they need trillions of dollars of revenue for AI they need it to do many layers more Microsoft just started a forward deployment engineer team to help organizations get the most out of AI guess what that's what you do when it doesn't work why do if this is so magical if this is so intelligent if this is so autonomous why do you need an army of McKinsey perverts to invade my business to make it functional the answer is it isn't functional LLMs are a con perpetuated by con artists and I'm sick of I'm sick of them I'm sick of the waste I'm sick of the people who have dedicated themselves to this graveyard smash I think it's insulting to humanity it's driving the cost up of everything and everyone involved should feel ashamed of themselves well on that note Ed Zitron thanks for [00:39:06] Speaker 2: taking the time thanks for having me if you enjoyed today's episode and you want to hear more of the tech report please consider liking and subscribing also you can get episodes of the tech report wherever you get your podcasts