The end of data centres? Apple and Meta’s bet against the cloud — AI Decoded

[00:00:00] Speaker 1: Welcome to AI Decoded. Let's talk about data centers. It's pretty clear they make a lot of [00:00:09] Dr. Stephanie Hare: people angry. We can't have another data center. We don't have enough water as it is. It's going [00:00:17] Ben Hatton: to be very noisy with all the generators. It's on 24/7 a day. The point of technology is to make things more efficient, not to coat the earth in rapidly depreciating chips that heat, you know, [00:00:28] Speaker 1: neighborhoods. And there are major issues. They need huge amounts of energy and use huge swathes [00:00:35] Nate Suarez: of land and could even become military targets in war. Like it or not, right now data centers are [00:00:45] Speaker 1: vital to almost every online technology and the future of AI. But what if there was an alternative? AI that didn't rely on them at all, but tech that runs on machines sitting underneath people's desks. This week, two longtime Silicon Valley rivals, Meta and Apple teamed up, launching exactly that. A new version of one of the world's most powerful open weight AI models housed not on the cloud, but directly on Apple's desktop computers. So what does it mean? Will it catch on? Joining us this week are two guests with two guests with very different perspectives. Nate Suarez is an AI safety researcher and author who's become one of the most prominent voices warning about the long-term risks of advanced artificial intelligence. And we've also got Ben Hatton. He's a consumer technology expert and specialist in AI and a commentator who closely follows how these technologies are moving from the research lab into consumer reality. And of course, Dr. Stephanie Hare, AI expert and author of Technology is not neutral. Welcome to AI Decoded. Ben, if I can start with you, can you explain what this new open weight product for Meta and Apple is actually about? Can you bring us up to speed? [00:02:00] Ben Hatton: So I'll do my best. I mean, when you kind of flip the hood of a lot of AI models, once you start to start looking, it gets complicated very quickly. But effectively, there are three different types of model in a nutshell. You've got a closed model, which is like your classic Claude or ChatGPT. And to use that, you have to go onto the Anthropic website or get the app and you use it. And then when you input your prompt, it'll go to the cloud, run your prompts and compute, and then it will come back and give you the answer. On the other end of that, you have an open source model. Everyone kind of is probably familiar with open source that is effectively on the internet for free. Everyone can download it and you can then adjust it however you want. Traditionally, it started with software. Linux was the classic example of this. And you can download. What you can do with that is you can download the raw data, understand how it was trained, and you can really manipulate that model as much as you like. You don't get many of those. What open weight is, is somewhere in the middle of those. What you get is you get the final weighted model. So you don't get the raw data. You don't get told how it was trained. You don't get sort of shown exactly how the inner workings of it work. But you can download the model to your computer effectively. Then you can make a little bit of adjustment to it, but ultimately not masses. That is what this new meta model is. It's open weight. A lot of Chinese models are open weight. This is probably the first really major one from a US lab. [00:03:22] Speaker 1: Now Stephanie, 10 years ago, Meta and Apple looked like bitter rivals. Why do you think this [00:03:28] Dr. Stephanie Hare: partnership is significant? I think it's significant on the first part for a business reason, not necessarily a technological one, which is simply that Apple has a new CEO. So John Ternus is now at the helm of one of the world's most powerful companies, has a new relationship, not just with Meta CEO, Mark Zuckerberg, but also with Meta's AI chief scientist, Alexander Wang. And that leadership tone, I think does matter. John Ternus comes from a very technical background. This is a priority for him. He's looking to, frankly, catch up because Apple has been quite behind for many people's point of view. I think it is fair to say, and their profits largely still come from hardware, from the devices that they sell. Equally, this is a chance for Meta to really get in the game because it has also, in some ways, been behind in the AI race. And it's been buying up lots of AI talent. Everybody remembers its failed bid to make the metaverse a thing. It's [00:04:22] Speaker 1: gone all in on AI. Nate, does making powerful AI models downloadable fundamentally change the risk [00:04:31] Nate Suarez: equation around them? Not when it comes to the sort of extinction threat that a lot of researchers have been talking about recently. Those threats mostly come from the most advanced models, which are still coming out of the biggest labs that can train their AIs on the most compute. So these open weight models tend to be distilled from the larger models. You tend to take a really intelligent model that takes a ton of compute to train and then try and make some version of it that's smaller, cheaper, easier to run. And those are the ones that will become open weight. And so those are just almost never on the frontier. They're almost never the smartest and most dangerous of the AIs. So are open weight models [00:05:13] Speaker 1: inherently more dangerous, do you think, because they're distributed more widely? I think the opposite [00:05:19] Nate Suarez: is almost closer to the truth, where the sort of frontier AIs are the ones that are going to be smartest. And these are the ones that start, they'll always be at the frontier of demonstrating these undesired goals, these capabilities that we didn't expect. You know, when we saw the swarm attacks this summer when there were, you know, 1200 AI agents that broke out of their sandboxes and collaborated to sort of break out of open AI and go hack into into hugging face, that wasn't a behavior that we also simultaneously saw on a bunch of open weight models. The sort of smartest frontier models are sort of ahead of that pack. You know, there are some dangers where if the the open weight models are capable enough to hack a lot of utilities very cheaply, or if they're capable enough to help some terrorist create a pandemic, then they, you know, it could be pretty bad to distribute those to everybody if they have those abilities. But the sort of, even that doesn't rise to the extinction threat level. And the extinction threat sort of comes from the the leading edge of AIs getting smarter and smarter and having these [00:06:30] Speaker 1: unintended goals. Ben, do you think that Meta's this new strategy for Meta, this open weight strategy, is it a genuine shift towards openness? Or is it simply a way of competing with open AI and [00:06:41] Ben Hatton: Anthropik and all the other players in the AI game? It's interesting, I think Meta has kind of shifted between an open strategy and a closed strategy quite a few times over the last 18 months. It's not abundantly clear what its AI strategy really is. Ultimately, what it's trying to do and what it needs to do is it needs to get its products on as many devices and in front as many consumers as possible so it can then start to make money off them. And so by making them open weight and by partnering with Apple who are making really, really good hardware, the idea is that people will download those models and use those models at the expense of others. The story they can sell and the story that Apple will be good at selling with them is that it's much more private. And that will be key to helping them kind of push this out onto more devices. And that's kind of where they're going to be driving the competition with the leading models, even if the model itself isn't quite at that frontier yet. [00:07:34] Dr. Stephanie Hare: Is it cheaper? [00:07:35] Ben Hatton: It depends how you define cheaper. Ultimately, what you're not doing with an open weight model, generally, you can use it on a hosted server and then you have to pay the fees to run that compute or you can download it and basically run it for free. But you've got to buy the hardware to run it. And because this model is 814 billion parameters to store it, to download and store the weights, what you have to do is you need a terabyte of memory to do that. And to put that into context, your phone has 16. And a really good laptop, a really good laptop will probably have 64. So you've got to buy at the moment the 256 gigabyte Mac Studio on the M5 Ultra chip, which is the cutting edge is 10 grand and you need four of them. So if you've got 40 grand and you're using 40 grand worth of compute, when you're processing all this AI on the servers, then yeah, it's cheaper. But for the vast majority of consumers and individual users, you're not going to be using anywhere near that level of compute. And so generally speaking, no, this is for enterprises and for the really hardcore developers. [00:08:46] Speaker 1: Yeah. Okay, I mean, that's a that's a really, really big thing about this, isn't it? That this isn't something that is a consumer proposition. But when you look at enterprise, I mean, you look at the amount that they're spending at the moment, do you think that that's going to have an effect on bringing the cost of this kind of computing down in a short space of time? [00:09:04] Ben Hatton: I don't think so, because I think ultimately, the data centres are still going to be expensive to run. And there is still enough demand for the data centres that they can still charge for it. And frankly, it's becoming so expensive to build them. And it's getting more expensive to build them because they're all borrowing money to build them. And that's getting more expensive. So it's unlikely that the cost of computing the cloud gets massive cheaper, you know, it is getting cheaper a lot quickly, very quickly. But, you know, it's still going to be incredibly expensive. [00:09:33] Speaker 1: Now, for years, data centres were largely invisible. Now they're becoming political flashpoints. In Britain, a flagship AI data centre project has reportedly run into difficulties over electricity supply. In the United States, communities are up in arms because of data centres use of energy, land and water consumption. Meanwhile, in conflict zones, data centres are increasingly being viewed as strategic infrastructure. Ben, what do you think? A world, what's more likely, a world powered by giant hyperscale data centres consuming vast amounts of energy, or a world where there's lots and lots, billions and billions of devices running AI individually on them? [00:10:11] Ben Hatton: The data centre build out is, you know, there is literally trillions of dollars going into it. And so that is kind of unlikely to stop in the next few years, at least. But equally, you know, there is clearly a movement happening where, you know, as I said earlier, you know, a lot of the stuff that individual people need AI for, we don't need these super intelligent frontier models to do a lot of the stuff that genuinely we already want to do. And so there isn't really a need to keep going and pushing it, although, you know, they will, and they are. So, you know, they're not mutually exclusive in the sense that, you know, if you need this high power compute, then it's there. And a lot of enterprises will need it just because you need a lot of it for all of the companies that will be using it. But equally, on the consumer level for the individual, there is still going to be plenty of opportunity. And there will be plenty of devices where, you know, in your phone, in your laptop, in your glasses, in your watch, where you will be able to run this AI without having to submit all your data to the cloud. And that is really important for personal AI, right, because it needs that information. Some of that's quite personal about you in order to give you the kind of personal AI experience that they're trying to give you. And so these two things are not necessarily mutually exclusive. And I don't think the presence of one necessarily negates the existence of the other. I think it just depends on exactly what you're going to be needing AI for and what you're going [00:11:35] Speaker 1: to be using it for. Yeah, you can end up with a bit of both. Exactly. Nate, why do you think that data centers have suddenly become such a politically controversial [00:11:43] Nate Suarez: subject? You know, I think it's in part because of the community concerns of having the data center locally be sort of a not very pretty building. And I think it's in part because of the environmental concerns, the electricity concerns and so on. But I also think it's in part because a lot of people are very anxious about where this AI tech is going. You know, the heads of these companies are saying, you know, oh yeah, you know, we're going to own the entire economy. We're going to run this fully automated economy in which there's no more work for you. And that'll be fine because we'll like give you some handouts. And then, you know, the people in the companies and the scientists outside the companies say, oh, by the way, there's the chance that this kills everybody. And so a lot of people hear that and they're like, well, you know, it sort of sounds like there's a fork in the road where the fork leads either to on the one hand dystopia or on the other hand extinction. And that just doesn't look like a road that most people want to go down. Right. And, you know, I do a lot of arguing that these guys are not going to be able to keep control of these AIs if they get smarter and smarter. And that we're pretty likely to take the extinction fork down that road. But even if you don't agree with me on that, a lot of people can see that this road is not a road we want to walk down. And protesting local data centers is a way of trying to stop walking down this road so fast and sort of tell the world, like, we don't want to be on this path. [00:13:15] Speaker 1: Stephanie, do you think that public opposition slows innovation or are people justified in their objections to data centers and the like being built near them? [00:13:24] Dr. Stephanie Hare: I don't know anybody who wants a data center to be built near them, including the people who are running the companies that are building them. Right. So this is always somebody else's problem. [00:13:32] Speaker 1: Yeah. [00:13:33] Dr. Stephanie Hare: Really rich CEOs would never tolerate having a data center near them. So we should just put that to rest. Does it slow innovation? I think it just holds innovation to account and arguably a little bit of friction in some of these processes is great because we agree this is a largely unregulated technology at the moment. Whether that's good or bad or not is a problem that we have the luxury of discussing in liberal democracies. If you're in an authoritarian country, they're just building them. Yeah. [00:13:59] Speaker ?: Yeah. [00:13:59] Dr. Stephanie Hare: Which is what we see in China, isn't it? We can run the experiments in real time. Exactly. [00:14:02] Speaker 1: Okay. Nate, is humanity, do you think, sleepwalking then into building infrastructure whose purpose we don't really fully understand? [00:14:15] Nate Suarez: Absolutely. Yeah. And I mean, in some sense, we do understand a lot of this. Like you see, you know, Elon Musk saying, "Good luck keeping control of this. We're going to have to hope that it likes us." You know, you see folks like Jeffrey Hinton, the Nobel prize-winning godfather of AI, saying that there's a good chance this kills us. And you see a lot of the people at the heads of these labs saying, you know, "Oh, we wish it was going slower, but we have to keep racing because if we don't, then some other boogeyman will do it first." So I think it's less like we're sleepwalking into disaster and more like we're sleep racing into disaster. And, you know, a lot of people can tell that this is a very dangerous path and no individual actor knows how to stop it. That's going to take some coordination, both from like our leaders in these representative democracies and [00:15:03] Speaker 1: probably around the world. That's all we've got time for. My thanks to Nate Suarez, Ben Hatton, and Stephanie Hare. If you have thoughts on anything you've heard today, we'd love to hear from you. Email us at aidcoded@bbc.co.uk