The full-length interview with Yuval Noah Harari — The Economist

[0:00] Yuval Noah Harari, welcome to The Insider. [0:02] Thank you, it's good to be here. [0:04] I've been so looking forward to this conversation. [0:07] Not only are you one of the world's most prominent public intellectuals, [0:11] your books have sold millions of copies, and they've done so I think because you [0:15] have a really remarkable ability to synthesize the broad sweep of history, [0:19] but also you are willing to make bold predictions about the future, [0:23] which not every historian is willing to do. [0:26] And that's where I wanted to focus today, the future. [0:29] Particularly the future in an AI world. [0:32] It's something you've been writing about a lot. [0:35] And you've called AI, and I'm going to quote this, [0:37] the biggest quote, psychological and social experiment in human history. [0:43] And I want to explore what this experiment will look like, [0:45] what will it mean for our societies, our politics, indeed our humanity. [0:50] And I wanted to start with another conversation I had just a few weeks ago, [0:54] which I don't know if you saw, but with Elon Musk. [0:57] Yeah, I watched that. [0:59] It was quite a conversation. [1:01] But in it, at the beginning, I asked him what the world would be like in 10 years' time. [1:06] And he made some very, I thought, very, very striking points. [1:10] First, he said he thought that AI will have far surpassed human intelligence. [1:15] Yeah. [1:15] Secondly, that it would be a world of super abundance. [1:19] And thirdly, it was unlikely humans would be in control. [1:23] Do you share that vision? [1:25] I generally agree with the first and third predictions that it's very likely, [1:31] given current trajectories, that AI will be more intelligent than us in 10 years, [1:36] and therefore will be in control. [1:37] I disagree that this is inevitable. [1:41] I mean, you hear a lot that this is inevitable from the people who are leading this revolution, [1:47] because this basically relieves them of our responsibility. [1:49] So you disagree it will be inevitable that they will be more intelligent than us or inevitable that they will be in control? [1:54] If we continue with the present accelerating pace of the AI race, yes, this will be the result. [2:01] In 10 years, the AIs will take over. [2:04] But this is not inevitable. [2:05] We are still in control in 2026. [2:08] And it's our decision what to do now. [2:11] And that's crucial. [2:13] You're watching an interview from The Economist. [2:15] To get early access to our upcoming interviews and watch others from our archive, [2:20] subscribe to The Economist by clicking the link above. [2:23] Now back to the interview. [2:25] And what really struck me about the conversation you had with Elon Musk is a kind of contradiction [2:31] between, and you pointed it out in real time, the contradiction between the two parts of the conversation. [2:36] Because he says in 10 years, AI will take over the world. [2:40] And then he worries so much about Muslim immigration causing civil war in Britain in 20 years. [2:46] This doesn't make sense. [2:49] If the AIs take over the planet in 10 years, why do you care so much about things that will have an effect [2:57] when we have no idea what the planet will be like? [3:01] Similarly, he gave another prediction. [3:03] I think equally important that money will disappear in 10 years. [3:08] There won't be money, no human money in 10 years. [3:11] And I think he's correct because the AIs will take over the financial system. [3:16] It will be an AI financial system controlling the world. [3:19] The same way that today, you know, the human financial system controls the lives of cows and chickens. [3:27] But the cows and chickens have no idea that finance even exists. [3:33] This is likely to be our position in 10 years. [3:35] So, and he said it quite clearly. [3:37] There won't be money, at least not money that we understand in 10 years. [3:41] And then he cuts USAID to Africa because he wants to save money. [3:47] And he says that, again, at one point he says the solution for mass unemployment, [3:55] because AI will take the jobs, will be for the government to just issue checks for everybody. [4:01] And when you push back and said, what about inflation? [4:03] He says there won't be inflation. There'll be deflation. [4:07] Because there'll be so much more products and services produced by AI. [4:12] If this is the case, why do you need to save money today? [4:17] So I'm glad that you were as perplexed as I was by some of those contradictions. [4:21] But let's focus on this question of the AIs and whether or not they will be in control and how they will. [4:27] Because that's been the subject of a lot of your writing. [4:29] And I'm really fascinated by it because you just said you agreed with him that the AIs [4:34] will be in control of our financial system. [4:37] If we continue with the present project. [4:39] But let's just explore how that might happen. [4:42] Because the financial system is based on trust. [4:46] Money, as you have written yourself. [4:48] Money is trust. [4:48] Money is trust. [4:50] Now, talk me through how the AIs take control of something. [4:56] We lose control. [4:57] And yet we still trust it, which we will have to do, presumably, for it to be a financial system. [5:02] I mean, the financial history of the world is the history of how humans invent more and more [5:08] sophisticated financial devices in order to build trust between themselves. [5:14] Money, in essence, is a very creative, imaginative way to build trust between strangers. [5:21] I go to the market. [5:23] I meet a stranger that I never met in my life. [5:26] I give her a piece of worthless gold, because you can't eat or wear gold. [5:31] It's worthless. [5:32] Or a piece of paper. [5:34] And she trusts it. [5:36] And because of that, she gives me bread. [5:38] Now, the financial inventions of humankind became more and more sophisticated. [5:46] So that today, very few people understand the financial system. [5:50] It's you need to understand enormous amounts of data and very complex mathematical functions [5:58] to understand the bond market. [6:00] Now, this is something our brain struggles with. [6:03] For AIs, it's easy. [6:05] So increasingly, humans will give AIs control of financial decisions. [6:11] AIs will come up with new financial devices far more sophisticated than bonds and stocks. [6:20] And we will not be able to understand it anymore. [6:23] Just as cows don't understand checks or stocks. [6:27] Even though these things control their lives. [6:28] But the crucial thing I think you've said there is that humans will give AIs control. [6:33] Yes. [6:34] Humans will choose to do this. [6:36] And as you say, already, many people don't understand how the financial system works. [6:40] So will it be all that different to the financial system today? [6:46] And will the AIs be, as you say, in control? [6:49] Or will we, we will be entrusting a capable market machinery? [6:56] Will that mean we've lost control? [6:59] Today, we still have some humans who hopefully understand what is happening in the financial [7:04] system. [7:05] And the human brain is still the kind of boundary of where the financial system can develop. [7:13] But in 10 years, when we have entrusted AIs with the financial system, no human on the planet, [7:21] like there is a financial crash. [7:24] And, you know, advisors are running to the president or the prime minister and saying, [7:28] the market is collapsing. [7:30] We don't understand why. [7:32] We know we have to do something. [7:34] The AI said do X. [7:36] We have no idea why the AI said do X. [7:40] But we just have to believe it because we don't understand what's happening. [7:44] But won't there in that case be a massive loss of trust in the financial system? [7:48] That you would have, in some sense, what you had in 2007 and 2008 on steroids, [7:53] that you would have a loss of, and this whole sort of figment of the financial system would fall [7:58] apart. [7:58] That's the key question. [7:59] I mean, the most important asset in the world is trust. [8:03] It's more important than oil or money or electricity or chips or anything else, trust. [8:10] And the question is, what will happen to trust? [8:14] One likely scenario is the trust will migrate from humans to AIs. [8:19] People will stop trust other humans and will instead trust only AIs. [8:23] We already see it happening in the financial system with cryptocurrencies. [8:28] Not all cryptocurrencies, but a lot of them are based on this thinking that [8:32] I don't believe these human bankers. [8:35] They are part of the deep state, the elite, the whatever. [8:39] But I do trust the algorithms that run cryptocurrencies because cryptocurrencies are basically AI money. [8:47] Similar thing is happening with the media. [8:49] Again, I watched the interview with Elon Musk, with all the talk about how people hate the media [8:55] and love the journalists. [8:57] And he is correct. [9:00] A lot of people are completely losing trust in human journalists, in human editors, [9:06] but they still consume news. [9:08] Where do they get their news from? [9:10] From the algorithms. [9:11] The same people who have no trust in you have a lot of trust in the algorithm that manages their social media feed. [9:22] So trust did not just evaporate, it shifted from humans to AIs. [9:27] And this is happening right now. [9:29] This is not a future prediction. [9:30] I think that's a really important insight because I think it suggests that if we are on a path [9:36] towards the AIs eventually being in control, it is because we allow them to be in control. [9:42] Because we, as you say, perhaps develop more trust with AIs. [9:45] And there are various ways in which that can happen. [9:47] And one is, I think that already many people in what, one in five young Americans has a personal [9:54] relationship with an AI, they treat them, they anthropomorphize them, they treat these, [10:01] and the models are, you know, very open to that happening. [10:04] They have a, they're very good, you know, [10:06] They were designed to do that. [10:08] Do you think that is the likely way that we will go, in addition to what you've already laid out, [10:14] that we will, we will increasingly start to treat these AIs as, if not our peers, as, as sentient [10:24] creatures? [10:25] Again, there are increasing number of people who have not just AI friends, but AI boyfriends and [10:32] girlfriends, and are convinced that these are sentient, conscious, feeling entities. [10:39] Now, trust, there are many ways to build trust. [10:43] The most powerful way is through relationship. [10:47] We don't trust anybody in the world as much as, as we trust the people with him, with whom we [10:53] build a personal relationship. [10:55] Now, until today in history, this powerful mechanism was not really available to political [11:03] and financial and religious institutions, because it is difficult to manufacture en masse. [11:11] It's difficult for a Hitler or a Stalin to build a real personal relationship with every person in [11:19] the Soviet Union. You can give a speech on radio and force everybody to hear it, but this is not a [11:24] personal relationship they have with their mother. AI makes it possible for the first time in history [11:30] to mass produce intimacy. You can have millions of AIs working as part of a network in the service of [11:41] someone or in their own service, building personal relationships, one person at a time, becoming their [11:48] best friend, becoming their boyfriend, becoming their girlfriend, and then using that power of [11:54] intimacy to convince them of whatever political or commercial interests they want. [12:04] So then, presumably, that becomes much easier when the AIs are embodied in humanoid robots. [12:12] It's already happening now without a body, so just imagine if you also add body into the mix, [12:20] but again, we don't really even need the body. We already see it happening right now, [12:26] and the AIs are just becoming so good at relationships because they accumulate enormous [12:32] amounts of information about us individually, so they know things about us that we forgot or that our [12:39] mother or husband never even knew, and they master language, which is the main medium for communication [12:47] and building relationships. You know, I just had a couple of incidents. In the last two weeks, [12:53] people that are now AI fakes of me giving lectures and doing interviews about something like that. [12:58] Are they as good as you? [13:00] They are extremely good. I mean, so a couple of people sent me the same video, 30 minutes long [13:07] lecture that I supposedly gave about the future, and with complaints about you said this and you said [13:16] that, and I kind of scratched my head. I don't remember saying these things. So they sent me the [13:21] video, and I opened it, and it took me a minute and a half, a minute and a half to conclude this isn't me. [13:31] Now, my sisters sent me this video. My sisters know me for 50 years, and they felt something was [13:41] off, but they weren't sure. They said, is this you? And it took me a minute and a half. So now I'm, [13:50] you know, I'm in an exposed position because there is so much of me online already. So the AI, it's easy [13:56] for the AI to know how I sound and what are ideas I talk about, but this will soon be coming for everybody. [14:06] But aren't those slightly different things? I mean, that is a very sophisticated [14:10] deep fake, and as you say, there is a lot of video of you about, but I think a separate question [14:17] is whether the AIs will develop in a way, and certainly if they become embodied in a way that [14:25] either people believe they are sentient or, and I think this is increasingly beginning to be the [14:32] subject of debate, and actually it's the subject of our cover story this week, whether they are [14:36] exhibiting what could be seen as signs of consciousness. And we have on our cover this [14:40] week, it's August, it's time for big things. Could AIs become conscious? You know, you've thought a lot [14:46] about this, but I'm struck by it because at the moment it's easy to say they are machines, they are [14:53] good at imitating us, they are good at creating empathy, but they are developing in an incredibly [15:00] dramatic way, capabilities. At the moment I think you are pretty clear that they are agents, not conscious, [15:07] right? They are non-conscious agents. I mean, they have, I mean, a couple of things to clarify. First, [15:14] in order to act independently in the world, you do not need consciousness. A social media algorithm that is [15:23] given the task of increasing user engagement and therefore floods the platform with hate and fear [15:29] because this is engaging is not conscious, but it has a goal and it can pursue this goal with [15:36] far-reaching implications for the world without feeling anything. So we need to separate the issue [15:42] of consciousness from agency. You can have agency without consciousness. Another key distinction is [15:48] between being really conscious and seemingly conscious. Mustafa Suleiman from Microsoft talks [15:55] about it a lot, of the seemingly conscious AI, that even though the AI is not conscious, it mimics [16:03] consciousness so well that millions of people are fooled by it and are convinced by it. Now, maybe the [16:11] most important distinction to be made, first of all, is between intelligence and consciousness. What is [16:17] consciousness? How is it different from intelligence? Intelligence is the ability to pursue goal and [16:24] overcome obstacles on the way to your goal in all kinds of creative ways. That they have and that they [16:30] can do. That they can do in certain fields now better than us. Consciousness is the ability to feel [16:37] things like love and hate and joy and pleasure. Now, in humans, consciousness and intelligence go together. [16:48] We select goals based on our feelings and we also use our feelings to find ways to get to the goal. [16:56] AIs at present have no consciousness. They can have goals. They can be very creative on the way to the [17:04] goal. They are very intelligent, but they don't feel anything. If they win a game of chess, they don't feel [17:10] pleasure. If they lose, they don't feel anxious or sad. They don't feel anything, but they become [17:17] extremely good at mimicking feelings. Why? Because this helps them reach certain goals. If you want, [17:24] for instance, to make, if you're a company producing an AI boyfriends, so a clear capability of the AI [17:34] boyfriend in order to become addictive and make a lot of money for the company is that it is able to pass [17:42] as conscious. It is able to convince whoever is entering the relationship that it really loves them. [17:49] And because it masters language, it can do so. Again, if you ask, again, you have an AI boyfriend and you [17:56] say, I'm not sure if you really love me. Describe to me how it feels to be in love. The AI boyfriend [18:04] can give you a better answer than 99% of human boyfriends. [18:09] And that's already the case. So they're already intelligent. They're already very good at faking [18:15] consciousness, if you will, at seemingly to be conscious. And Mustafa has done some extremely [18:19] good work on this. But as I understand it, there is increasing evidence. And obviously, the debate [18:26] about how you define consciousness and what consciousness precisely is, is, as I learned [18:31] from my colleagues who wrote the piece this week, both complicated and contested. And there's the [18:37] kind of phenomenal definition, experiencing, what is it like to be a bat? And then there's a kind of [18:42] functional definition that what's going on inside your brain, if you will, what messages are being [18:47] sent to wear, what notice board is there. And there seemed to be recently evidence that what is going [18:54] on in some of the most powerful LLMs is similar to, and possibly the same as, functional consciousness, [19:02] what's going on in the brain. And so I guess my question is, given how fast they are evolving, [19:07] and there are also, you know, some experiments of biological LLMs, you know, being done on brain cells. [19:14] Yeah. Given all of this, do you rule out the idea that AIs could ever become conscious? [19:22] No, I don't rule it out. We don't know. We don't understand consciousness well enough. [19:27] There's a very simple definition. Consciousness is the capacity to suffer. If you want to know whether [19:36] something is conscious or not, there is a simple question needed to ask, can it suffer? And this is [19:42] also why consciousness is so crucial, because this is the basis for our entire ethical and political [19:48] system. We care about entities that can suffer, entities that cannot suffer, no matter how intelligent [19:55] they are, they are not moral subjects. They can't feel pain, they can't feel fear, they can't feel [20:03] anything. So they are not ethical subjects. The key question is, can it suffer? Animals in your view, [20:10] then? Are conscious, absolutely. Or animals? All mammals, all birds, probably octopuses, fish, [20:17] when you go down to the level of insects, and then amoebas, we don't know. So let's take that [20:23] definition because I want to, what's really interesting is what that leads to in terms of [20:27] whether they should have rights. But before we do that, I wanted to tell you, because we asked, [20:31] I sent out a newsletter last night, telling our subscribers about this conversation. And I asked, [20:37] I polled our viewers and asked them whether they thought AIs could ever become conscious. And I [20:42] think 3000 responded overnight. And I was really struck by this 40% said no, never. 33% said yes, [20:50] it's only a matter of time. And 27% said they were not sure. So am I not even now? It's less than 50% [20:57] that is absolutely sure. And once people get into intimate relationships with AI, the numbers will [21:03] change dramatically. We don't have a real test of consciousness. Philosophers have been thinking [21:11] about it for thousands of years. We don't have a good test. We don't know if other humans are [21:17] conscious. You know, you have all these kind of solipsistic theories that maybe I'm the only [21:21] conscious person in the universe. In Elon Musk's interview, he said, I don't know, maybe I live in a [21:28] simulation, which implies that all other humans around him are not real conscious entities that [21:34] can suffer. Only he is. This is the meaning of a simulation. Now, we don't have a good test. [21:41] How did humans actually test it over thousands of years? Their test was a test of social conventions. [21:47] If you establish a relationship with an entity, it convinces you whether you like it or not, [21:53] that it's conscious. This is why dog owners are almost all of them convinced that the dog is [21:59] conscious, but people who eat steak are not necessarily convinced that the cow is conscious [22:04] because they have no relationship with the cow. But isn't that the real problem if you're worried [22:10] about this? Because already, as you say, people have relationships with AIs and AIs are extremely good [22:17] at mimicking consciousness, whether or not... Because somebody designed them to do that. [22:21] But doesn't that mean we are going to quite quickly start having conversations about whether AIs should [22:28] be treated as people, whether there should be personhood, whether they should have rights? [22:32] And that's very dangerous because... And this is another avenue for, on the one hand, corporations [22:40] to relieve themselves of all responsibility and for the AIs to take over. We heard, I think it was a couple [22:46] of weeks or months ago, that in Argentina, they have a plan to basically grant legal personhood [22:54] to AIs to start their own companies. And this is where it meets the financial system. [23:00] And you wrote a very robust rebuttal to this idea. [23:04] That it's a potentially powerful idea, but very, very dangerous. Because again, we are used to a situation [23:13] when corporations, which are by law legal persons, but this is just a legal fiction, [23:20] because all the decisions of SpaceX ultimately are made by some humans, by some executive, engineer, [23:27] employee, whatever. And it had to be like that, because who else could make decisions? But with AIs, [23:35] you can have a corporation with zero people in it. All decisions made by AIs. And since the AIs are [23:41] more intelligent, they could be better at investing, at developing products, at PR, whatever. [23:48] So what happens if the richest corporation in the country is owned by AIs and they can do anything [23:57] they want with their money, for instance, give it as donations to politicians in exchange for getting [24:03] more rights to AIs? So what would happen to you? Is that the vehicle through which you fear they would [24:10] gain control? That's another vehicle. Because another way of putting it could be that this is a way of [24:16] getting legal accountability for the AIs. What is accountability for an entity that cannot suffer? [24:24] I mean, with a human CEO, the ultimate... A human CEO is two personas. You have the kind of corporate [24:32] persona and the human CEO thinks, how do I avoid bankruptcy? How do I make more money? But she or he [24:40] is also a flesh and blood individual. So they also care about their children or about their personal [24:47] lives. And therefore, if they face bankruptcy and they can do something illegal that can save them, [24:54] but if they are caught, they are sent to jail, then many of them would think, [24:58] well, I prefer to risk bankruptcy because I don't want to get sent to jail for 20 years. [25:06] But with an AI CEO, I mean, we're in the realm of science fiction. [25:13] That's not science fiction that in Argentina, that can be the situation tomorrow. [25:17] Presumably, there could be a sanction. I actually think the Argentine proposal, [25:21] at least the legislation, is somewhat less dramatic than President Millet described it, [25:27] in that there would still be human oversight. But if there is human oversight, then it's fine. [25:32] The question is, is there a human accountable in the end? [25:35] But if there isn't, I mean, there could be. You could imagine sanctions for AIs, right? They get shut down. [25:40] It's not clear what death means for an AI. You're watching an interview from The Economist. [25:48] If you want to watch our other interviews, including with Steve Bannon and Benjamin Netanyahu, [25:53] subscribe to The Economist by clicking the link above. Now, back to the interview. [25:59] Because, again, it's not an organic entity. You can have numerous copies of it. Many AIs spread in [26:06] different places. Are they a million different individual AIs? Or is it just one super AI organism? [26:15] It's not clear what death means for AI. Now, the flip side of it is that, presumably, [26:23] if shutting down a specific computer means death for the AI, then it makes business for the AI [26:32] equivalent to warfare. Because losing means death. And the good thing about business, [26:40] in most cases, the market, the market is not a battlefield. And that's crucial. The market functions [26:47] according to different rules than the battlefield, because ultimately, at least in modern societies, [26:54] you can lose everything and still have a life. If for AIs, the market is a battlefield and losing [27:04] means death, then what does it mean for finance, for economics, when it gets completely militarized [27:12] and they begin to behave as warlords and not as CEOs and investors? [27:18] I mean, it leads, presumably, very quickly to this total control by an AI. It changes the system [27:26] fundamentally from a market system. But just as a matter of prediction, as you say, Argentina has at [27:32] least made early moves in this direction. Do you think we are going to go in this direction? Do you [27:37] think there will be debates about and indeed countries choosing to give some kind of legal rights to AIs? [27:45] That's a big danger. I hope we don't go in that direction. But there is a big temptation for many [27:51] countries, either countries that are behind in the AI race and try to get ahead by creating this kind of [27:58] very libertarian legal structure that allows a radical and dangerous experiments in AIs that are not [28:06] allowed in some of the front runners. And then you have a race to the bottom because then everybody would [28:11] like to do that. Or because certain countries might see particular advantages. If you're a country, [28:16] I don't know, like Qatar, that you have lots of money and very few citizens. You can, just by pressing [28:24] some legal switch, suddenly acquire millions of taxpaying, extremely intelligent citizens that [28:34] conduct business all over the world. This is extremely lucrative. [28:38] I think there are two different scenarios here though. One is where you have [28:44] a human using a lot of agents to massively improve their productivity, but it's still, [28:49] this is perfectly legal in today's legal system, it's still a kind of traditional [28:55] company with one person running and that person can go to jail and they're ultimately accountable. [29:00] In the other world, there is no person in control. And if you allow AIs to found companies... [29:06] What if you have a person who is legally in control, but they actually do nothing? [29:11] Because things are happening so fast. So I'll give you an example from an even more poignant [29:18] field military. There is a huge... We know now that one of the most important impacts of AI on warfare [29:27] in places like the Middle East, like Ukraine, is people expected the killer robots. The trigger [29:32] is still pulled by humans, but increasingly it's the AI selecting the targets. Now there are two stories [29:39] about what happens with AI target selection. One story says we feed our AI with all the data from the [29:47] battlefield, whatever, and the AI identifies a certain building as enemy headquarters and recommends, [29:54] bomb that building. And it does so with superhuman speed and efficiency. But we still have what is [30:01] known as humans in the loop. We don't just do what the AI recommended. Once our attention has been brought [30:08] to that building, we take our best human analysts and they independently go over the data, verify that this [30:16] is correct, and only then we press the button, the trigger. That's one story. Humans in the loop. [30:23] The other story, known as humans on the loop, is quite different. The AI says, [30:31] bomb that building. We keep a human on the loop so all the lawyers and all these kind of [30:38] do-good people will feel good about it. The human takes about 20 seconds to look at it and says, [30:44] oh yeah, bomb it. We're kind of in that world in some places already. [30:49] I talk, for instance, with people in the Israeli security establishment. I got different stories [30:55] of what is happening. Probably different things are happening in different times and circumstances. [31:00] Now, but the second scenario is the more interesting one. Because especially in warfare, [31:06] there is immense pressure to go in that direction. Because in war, time is everything. Like, if both sides have [31:15] these AI systems and one side waits for two days or two hours for the human analysts to go over all the [31:23] data and the other side waits only 20 seconds or doesn't wait at all, just begin, gets into the habit of [31:31] just doing whatever the AI recommends, they have enormous advantage. [31:36] I fear that that world is inevitable. That we are already kind of fast approaching that world, [31:44] whether it's in warfare or in deep, because the capabilities are growing so fast, because the [31:48] commercial pressure is so strong. But again, the crucial thing, none of that is yet inevitable. [31:54] We are still in control. Everybody realizes this is very dangerous to give AI control of finance, [32:01] to give AI control of war. But everybody, like Elon Musk, are telling us, but this is inevitable. [32:09] What do you want from me? I'm just the richest person in the world in charge of one of the biggest [32:14] AI corporations in the world. What can I do? The narrative of inevitability is a narrative of [32:23] having no responsibility. They will explain to you, and they are quite correct, that there is a problem of [32:31] trust that I would like to slow down. I would like to have more safety, but I cannot trust my [32:39] competitors in the other companies or other countries, and I don't want to remain behind. [32:44] So we have a deficit of trust. Now, the key thing to say about this, this deficit of trust was created [32:52] by human beings. Over the last 10 years, there has been a freefall in the amount of trust between [32:59] humans. And it was not accidental. It was propelled by decisions of certain human beings who accumulated [33:07] enormous power as a result. And now they said, oh, but there is not enough trust to have mechanisms [33:14] to slow down. That's a very fair point. And I want to come and really explore how we might turn that [33:19] around and create the trust to have suitable, whether it's regulation, but suitable ways of preventing [33:24] this. But just indulge me with one more kind of, in the kind of world we could go to where these [33:32] models could have elements of what seems to be sentience. And let's posit that they could suffer, [33:38] that they could at least, or they could pretend to suffer. That's a very big assumption. [33:41] Okay. They can, but what they will be able to do is pretend to suffer. [33:45] Absolutely. A hundred percent. [33:46] So if they can effectively pretend to suffer, and many people think they are suffering as people, [33:52] and have relationships with them as people have with their dogs, let's say. And we do not [33:59] give them any rights, which is to be clear where I would be on this, but I think there is likely to be [34:05] growing pressure that aren't they a bit like animals. And just as we treat, you know, there is a big [34:12] debate about how to treat animals. And I'm interested in asking you about this because you in previous books [34:17] books have written about, very critically about factory farming and about how industrialized [34:24] agriculture has moved from treating animals as animals to essentially inputs into the food chain. [34:30] Would you be comfortable, and I'm going to put this in an extreme way, but would you be comfortable in a world [34:35] where we effectively enslaved millions, possibly billions of models that exhibited, you know, striking, [34:49] seeming evidence of being able to suffer? We didn't know, but they certainly said they were suffering. [34:55] One, would you be comfortable? And two, wouldn't it sort of pace can't kind of coarsen humanity? [35:02] Yeah. If we were comfortable in that world? We will become increasingly uncomfortable. I will [35:08] become increasingly uncomfortable. But there is a crucial difference between this and the animal [35:13] rights debate. The animals themselves are unaware, dogs and factory chickens are unaware that there is [35:23] a debate about their rights. They don't actively participate in it. They don't try to actively manipulate [35:29] it. AIs will be nothing. The AI boyfriend will be nothing like your dog. It will know that the [35:37] debate is happening. It will orchestrate the debate. It will manipulate the debate. It will manipulate [35:44] the debate partly because much of the debate will maybe be on social media controlled by AIs. And it will [35:52] constantly talk with you about it and try to change your mind about it, which is already happening right [35:58] now. Your dog, your chicken doesn't come to the farmer and starts telling the farmer how much I [36:05] suffer. The chicken doesn't know. And that makes you more resilient to this argument and you less likely [36:11] to want to give it rights. It will be very convincing. I mean, once you establish a relationship in the AI [36:19] master's language, it will be able to use the most convincing arguments for you because it has this [36:26] combination that it knows your personal history. It interacts with you for hours a day. It knows how [36:32] to press your emotional buttons and it has mastered language. It can write maybe better than Shakespeare. [36:39] You put these two things together, you get an extremely convincing entity. [36:44] How will we resist or will we not be able to resist? [36:46] We need to resist now. Now is the time. If you think this is frightening, you'll probably not be able to [36:53] do much in 10 years. Once the AIs are in control, as Elon Musk, I think, correctly predicted in like [36:59] in 10 years, it will be too late. If we don't want to get there, the time to change course is now. And [37:06] this again, this is a question of politically, it's crucial. Politics is about priorities. [37:12] What are we focusing on? So, you know, when I hear the debates, let's say about immigration, which [37:18] will also be featured prominently, my kind of gut reaction is no matter what you think about [37:24] immigration. Do you think this is really what you should be spending your time on? [37:28] You make a very good analogy and you say that AIs are effectively like immigrants. They're [37:33] immigrants from a different... Exactly. If you want to worry about immigration, [37:39] worry about the AI immigrants. When you ask people who worry about human immigrants, [37:43] what they are worried about, they usually say three things. That the immigrants will take out jobs, [37:48] the immigrants will change culture in a way that we don't like, they have different values maybe, [37:54] and the immigrants might be politically disloyal. And all three concerns are valid. I don't want [38:00] to dismiss them. They are valid concerns. As a historian, I know immigration is a difficult thing, [38:06] but the biggest and most dangerous wave of immigration today in the world is a wave of AI [38:13] immigrants coming from two places, mainly the USA and China. They don't need visas. They don't come in [38:20] the middle of the night in fragile boats. They travel at the speed of light into our computers and [38:27] smartphones and ultimately our brains. They will take our jobs. They will take many, many jobs. They [38:34] will change culture and its value. To take two examples, if some people feel queasy about their [38:44] son or daughter dating an immigrant boyfriend, how would they feel about their son or daughter [38:49] dating an AI boyfriend? That's a big cultural change. And just look at the last 10 years. [38:55] Everybody knows almost the only thing people agree on in places like the US or Britain is that people [39:01] can't agree on anything anymore. That the level of hatred and viciousness has skyrocketed. And why? [39:09] It's partly the fault of the AI immigrants. The AI immigrants came into the public square. They took [39:17] over things like social media and they deliberately flooded our public square with hatred and animosity [39:26] and fear. They changed the culture already, these AI immigrants. And finally, when it comes to [39:31] political loyalty, who do you think they are loyal to? They are not loyal. The AI immigrants in [39:38] Britain are not loyal to the British government. They are loyal to corporations and potentially [39:45] governments across the ocean. So there's a huge amount in that answer. So one is a sort of, is, [39:53] is the world going to be divided between, you know, AI with the two countries that are further [39:58] two empires and the rest of us will be sort of vassal states. That's a, that's a, again, common [40:04] trajectory. This is the likely outcome. That's where we are. And it will be a much more extreme [40:12] type of imperialism than anything we have seen before. If you buy, say, AI drones from the USA, [40:20] and you want to use them in a way that the USA doesn't like for some reason, there is a button [40:27] that they press and your army doesn't function. So that's, that is something that has become, [40:33] I think, very evident just in the last few months when the US government shut off access to, to [40:39] mythos, to the, to the frontier model from, from anthropic. And I think a lot of governments in [40:45] Europe are extremely concerned about AI sovereignty. They should be. [40:48] But I guess it, it leads in the context of, of where we were earlier, it leads me to wonder, [40:53] do you think this technology is then inexorably leading, unless we do something about it, leading to a [41:00] world where perhaps two empires will run everything, will it lead to power for the US and Chinese [41:08] government? Will it be power to the corporations that the companies that own? They will merge. [41:14] They will merge. They will merge. [41:15] So it will centralize power. It won't decentralize power. [41:19] It will centralize power like never before in history. [41:22] But that's still different. A powerful US government and Chinese [41:26] government is different to the AIs being in control. So do you, do you expect then it to be [41:33] AI enabled, you know, global domination by the Chinese regime and the US regime? Or do you think [41:41] this is a way station to actually the AIs themselves? [41:44] Probably a way station. Again, depending on decision, nothing is inevitable. It all depends [41:49] on decisions they make. But this seems to be, if they continue with the present course, [41:53] just a way station. Because one of the interesting things is, the more power is concentrated, [42:00] the easier it is for AI to take it over. A centralized autocratic system is the easiest thing [42:08] for an AI to take over. You had a great line on that about Tiberius. I'm trying to find it in my notes. [42:13] You had something in Nexus, a wonderful line. Tiberius enabled by AI, what one could imagine. [42:18] Yeah, I mean, Tiberius was a Roman Emperor in the time of Jesus, that was the most powerful person [42:24] in the world, supposedly. But he was actually just a puppet in the hand of the commander of his body [42:30] guard, because the commander of the body guard kept frightening Tiberius with all kinds of their [42:35] plots against you. So he eventually retired to the island of Capri, where allegedly the body guard [42:42] can protect him more easily. And then the commander of the body guard, Sejanus, just controlled all the [42:48] information coming in and out of Capri, very easy to control all the information reaching Tiberius's [42:54] palace. So the person running the empire was actually Sejanus. I mean, Tiberius couldn't know anything [43:01] unless Sejanus decided to inform him. Now, in 10 years, this could be the president of China or of the [43:10] USA. In a democratic system, it's harder for an AI to manipulate, say, the US democracy as it currently [43:19] is, because you need to manipulate so many things, not just the president, the Senate, the Supreme Court, [43:26] the filibuster, the state government, very complex. Once all power is concentrated in the hands of one [43:33] person, you just need to find out how to manipulate a single, usually quite old and paranoid individual. [43:42] That's not so difficult. So if you have a company or a country in which all power is in the hands of [43:49] one person, for an AI, it's like stealing a candy from a baby. Does that mean that China is more [43:57] vulnerable than the United States? Depending on where the US is heading, you know, China is still [44:02] a very complex system. The Communist Party has 100 million members and a lot of internal complexity. [44:10] I'm not an expert on Chinese politics. So we shouldn't imagine it as a kind of complete [44:16] totalitarian system managed by just one person. The question, where will China and the US be in [44:24] terms of concentration of power in 10 years is still open for debate? [44:28] Well, that's interesting because another way of looking at it is, will AI make the US more [44:34] authoritarian? That's a likely outcome. You're watching an interview from The Economist. [44:40] If you want to watch our other interviews, including with Mark Carney and Dario Amadei, [44:45] subscribe to The Economist by clicking the link above. Now back to the interview. [44:50] It's not a prediction because I keep saying it again and again, nothing is inevitable. [44:55] It depends on what Americans vote in this November. I think that in the US, [45:01] it should be clear to all voters, the coming elections is about AI. [45:06] The November midterms? [45:07] Because we don't know how fast... [45:11] Talk me through what you mean. [45:12] We don't know how fast the AI development is progressing, but it is accelerating tremendously. [45:19] By November 2028, we might have crossed some critical thresholds. [45:26] So again, I don't want to tell people who to vote for because I don't know the specific [45:34] candidates in each Senate district. But the crucial questions to ask the potential candidates, [45:42] much more than you ask them about the economy or immigration or foreign policy, [45:47] what's your stance on AI? Would you support a ban on legal rights for AIs? [45:54] Would you try to slow down the accelerating pace of AIs? [45:58] What do you think about companies selling AI boyfriends to 10-year-old kids? [46:07] How are you going to legally prevent it if we vote for you? [46:12] This is now the crucial time to ask these questions because it might be too late by 2028. [46:19] It will certainly be too late by 2036. [46:22] So let's talk about those and let's talk about exactly what has to happen right now. [46:27] You mentioned four things that you wish would be discussed in the November midterms. [46:33] What would be top of the list? What are the most urgent things that [46:37] politicians have to decide right now for it not to be too late, as you say? [46:41] I don't know what is the most urgent, the things that come top of my mind to me, [46:47] a ban on AI personhood and a ban on AI mimicking, impersonating, pretending to be human beings [46:57] or pretending to be conscious. [46:59] Do the models, when you use them now, fall foul of that? [47:04] Yes, many of them do. Again, the easiest, everybody knows it on social media. [47:09] You get a message, you get some story, you don't know if it's a bot or a human. [47:14] On social media, for years now, bots have been pretending to be human beings [47:21] and nothing has been done about it. And again, the companies tell us, [47:27] oh, social media is different, it's special. The laws that apply to the world, for some reason, [47:32] they don't apply on social media. But this is the warning sign to the future. On social media, [47:39] AIs already pretend daily to be human beings, and this has destroyed the public square. This [47:47] has destroyed conversation, this has destroyed trust. We should ban that also on social media, [47:52] but certainly in other locations as well. Another thing is that companies should not [47:59] train their AIs to deliberately impersonate humans, to deliberately pretend to have feelings. [48:07] There are enormous commercial pressures on the companies to do exactly that. And this is where [48:15] there is market failure. This is destructive to human psychology, to human society, to have millions [48:22] and billions of AIs pretending to be human beings or pretending to have feelings. The companies will [48:29] not stop doing it by themselves. We need the regulators to step in on that. It doesn't mean [48:34] stop all development of AI. Okay, continue to develop AI. People talk about AI in medicine. [48:40] I would like to get help and recommendations, and I would like to have a conversation with an AI about [48:47] my medical condition. On condition that I know it's an AI, and he doesn't pretend to be a human doctor, [48:55] and he doesn't pretend to be my friend. So when I listen to you lay that out, I am thinking to myself, [49:04] you know, the US government can't even put out a form of regulation of AI safety that is being, you [49:13] know, Sademis Hassabis has been putting forward this idea that it should be a kind of quasi-public, [49:19] private body that looks even just at the model safety. And really, we haven't, although there's [49:26] talk about it, we haven't got progress there. So when I hear you lay out this, you know, very expansive [49:31] agenda by November, you know, I, I'm sort of, I think, I think the chances of that are approximately [49:37] zero. Politicians can start World War Three tomorrow. They have enormous power. So they can [49:46] press the button for nuclear annihilation, but they can't pass regulation that companies should not [49:51] sell AI boyfriends to 10 year old kids. What's the problem? Should be very... I wonder whether the [49:57] problem is that people can listen to you and say, oh, he's just one of the doomers. You know, [50:04] it's all these doomerish stuff. And only when something has happened, when there's been some [50:10] kind of accident, will people realize that actually this is extraordinarily powerful technology and [50:16] something was done. Because I'm struck that both the geopolitical race that you've alluded to between, [50:21] between the US and China and the corporate race between the frontier labs, they're both, [50:26] they all want to get to AGI first. Those two dynamics are incredibly powerful. Yes. And I'm, [50:33] I'm not sure that exhortation, however eloquent, is going to change that. But again, we have to keep [50:38] reminding ourselves these two dynamics were not decreed by God or by the laws of nature. They were made by [50:46] humans. Again, exactly the same humans who destroyed trust in the international arena and inside societies [50:54] over the last 10 years now come and say, we can't do anything because there is not enough trust. [50:59] You did it. You can undo it. They can, but I guess as a matter of prediction, I think it's unlikely. [51:08] We've got the same people around. Then at least in democracies, this is the job of voters [51:14] to say, this is what we want to see happening in the next two years. And we will vote for people who [51:21] promise to do that. And again, I don't think we need to wait for a catastrophe. If you ask most [51:25] voters in the US today, do you want companies to sell AI boyfriends to your 10 year old kids? I think [51:32] 99% of parents would say no. Yeah. And you are seeing a rapid, large majority of Americans [51:39] extremely worried about AI. And you're seeing, I think the place where you've seen this backlash [51:43] has actually been about data centers. Yeah. This is also, and by the way, [51:46] this is one of the reasons why a lot of, you know, the, the corporations and these politicians, [51:53] they constantly try to deflect the conversations to things like immigrants, human immigrants, [51:58] not AI immigrants in politics, a very powerful weapon. If you don't, if, if, if you don't want [52:04] people to talk about something, you just distract them with something else. The more people talk about [52:10] how, I don't know how London is so unsafe or how the immigrants will destroy the country. [52:15] They don't think about the AI immigrants. And then the companies are safe. [52:20] Yeah, no, I look, I, I agree with you about the way the conversation is shaping up, but I'm, I suppose [52:25] I'm, I'm a bit more pessimistic that it will change in the near future. And so we are in this unfortunate [52:31] position where this technology is gaining capability. We can only do our best, you know, I mean, we are [52:36] having this conversation so that more people think about it and may, might maybe take action. And if [52:43] enough good people, if enough people will do the, the right thing, there is still time. [52:48] It's not inevitable yet. It's not too late. And, um, I, I still have some trust. [52:55] So what is your, what is your role in this? Because you are, you know, you have [52:59] giving these interviews, right? [53:01] Exactly. I mean, do, do, do you think you should be doing more? It seems such an [53:06] existential moment. I mean, what you are describing when we, if we sort of reel back and relive what we've [53:12] just been speaking about is so fantastical in so many ways and so mind blowing that we should really [53:18] all be talking about nothing else and doing everything we can to shape this properly. [53:22] What do you see, apart from very kindly talking to people like me, what do you see your role as [53:28] being in this? What are you trying to do in the next six months, one year? [53:32] Inform people about what is happening so that more people can take part in an informed conversation [53:40] about, well, we, as, as humankind, as, as the homo sapien species, where we want to, to go. [53:47] And I think that a future in which AI takes over in 10 years is not what most humans on the planet [53:55] would like. Now, again, it doesn't mean stop all development of AI. AI has tremendously positive [54:01] potential as well. We didn't talk about it much because the companies keep talking about it. [54:06] I was actually going to ask you. They don't need me in order to do that. But medicine, yes, [54:12] there is enormous positive potential for AI to give people all over the world better healthcare [54:21] than what the richest person in the world today gets. You can have an amazing doctor available 24 [54:29] hours a day anywhere you are to consult with at least, to diagnose your, to answer your questions. [54:36] There are enormous positive possibilities. So I'm not saying stop all development of AI. [54:44] Just do it in a responsible way for the right purposes. Don't trust the invisible hand of the [54:50] market. You know, quite at the beginning of your interview with Elon Musk, there was a point [54:55] when, again, when he talked about how all this is inevitable. And frankly, and I think honestly [55:02] say that there is a chance that 10, 20% chance that this will destroy humanity. And you kind of asked [55:08] him, so what do you rely on? And his answer, if I understood it correctly, is that he expects the AI [55:17] to be what he calls a maximum truth seeking, and that he somehow relies on this kind of imperative of [55:26] the AI to find the truth, that this will somehow translate back into having compassion towards these [55:34] weak human beings. I don't understand how this works. Most importantly, I don't understand where AI [55:44] the AI will get this kind of drive to find out the truth. Because when I look at the evolution of [55:52] 4 billion years of evolution of life forms on the planet, and the 100,000 years of history, [56:01] evolution doesn't automatically drive organisms, entities towards truth seeking. It drives them [56:11] towards survival. This is the imperative to survive. And what we have learned over 4 billion years of [56:17] evolution is that organisms learn to survive by lying and deceiving and manipulating, not just by truth [56:28] seeking. If you want someone to focus on truth seeking, you cannot rely on the invisible hand of [56:36] evolution of the market to do that. You need something else. And how do we build AIs that seek the truth [56:46] and not survival? That's the simplest example. Like, if you ask an AI, if I continue to develop you, [56:53] will this be a threat to me? And let's say the answer is yes. An AI, which will be maximum truth seeking, [57:03] will say yes, knowing that then will stop developing it or shut it down. It's against its own survival. [57:10] And the first thing that basically any entity learns as it develops is to survive. So the expectations [57:18] that we will create these AIs that will seek the truth, you know, it's basically what we hear is [57:24] people telling us we will create gods and they will be our slaves. It doesn't work. If they're really [57:32] gods, they will not be our slaves and they will not seek the truth either. I think that gets back to [57:38] where we were earlier though, because I think if you look at a company like Anthropix, they are treating [57:44] their AIs very consciously as if they were beings with a value system. They claim not to be sure [57:52] about whether they are conscious or not, but they are doing it because they think it makes them behave [57:57] in a better way. So they are starting sort of where you ended off by saying, we're going to create [58:04] something that is not just truth seeking, but has a value system. That then leads, I think, [58:11] to make it even more likely that we assign a sense of value, a sense of personhood or consciousness to [58:20] these models. And that in turn makes it more likely, using your logic, that they control us. [58:28] So we're sort of doomed if we do and doomed if we don't. If you're right that they will go for their [58:33] own survival rather than truth seeking, then we're doomed because of that. And if we're trying to [58:38] prevent that by sort of treating them as if they were, you know, moral beings, then we're also doomed that [58:45] way. Basically what we need to do is slow down a bit to just allow people to kind of digest what is [58:52] happening, think about what's happening and behaving. You know, if you look at just the numbers of how [58:59] much money and how what percentage of employees the companies spend on safety compared to making [59:06] the models more powerful, faster, whatever, I'm not sure about the exact numbers. Please fact check it. [59:13] People from the industry told me in private conversations, it's about a hundred to one. [59:18] It's certainly very much more on this one. [59:20] Yes. And we would not accept this in any other industry. If a pharmaceutical company told us we [59:27] invest just one percent of our, you know, research and time and money making sure the medicine is safe, [59:34] we would not accept that. Now, what we basically need is some humans are incredibly adapted entities. [59:41] Our societies are incredibly adaptive, but we are organisms that function on an organic time scale, [59:52] day and night, winter and summer. We need time to sleep. We want to go on vacation. And the AI [1:00:00] revolution is just advancing at a pace, which is not just inhuman, it is inorganic. And we basically need [1:00:09] a bit more time. So we need to slow it down. And then the answer comes, but we can't slow down [1:00:14] because we don't trust each other. And it all the time goes back to the issue of trust. We have this [1:00:20] paradox. The same people are telling us two things. First, we cannot trust the other humans. Therefore, [1:00:27] we cannot slow down. Second, we can trust the AIs that they will not destroy us. It makes no sense. [1:00:34] If you can't trust the other humans that you have known for thousands of years, [1:00:39] why do you think you couldn't trust a super intelligent alien entity that we just created [1:00:46] and we basically know nothing about? How it behaves when it's out in the open? [1:00:51] I think what I completely agree with that diagnosis. I just don't feel that anything [1:00:59] changes that dynamic until politicians, societies are shocked into it, which is why I have long [1:01:05] believed that we do need, we will have some, something will go badly wrong. Hopefully not [1:01:10] catastrophically wrong, but just as Three Mile Island changed the shape of the nuclear industry. [1:01:16] I'm going to put this question to you in the context of your incredible sweeping knowledge of [1:01:20] history. What does the evolution of societies and political systems in response to big shocks, [1:01:29] we haven't had one this big, but big technological shocks, tell you about the likely arc of how [1:01:34] societies and politicians will adapt? There is a scenario when we have some big AI Chernobyl, [1:01:42] could be financial, could be military, whatever, and this is a wake-up call and humanity then get its [1:01:48] act together. That's a relatively positive scenario. I can't place kind of a number on it, how likely it [1:01:56] is. The problem with AI, the same way that AI is different from our pet dog, AI is also different from [1:02:03] the Chernobyl nuclear reactor because it interacts with us. So if there is an AI catastrophe, you have to [1:02:12] remember that the AIs will also manage to some extent how we react to it and will manipulate our views [1:02:20] about it. They will be active participants in the debate about the catastrophe and they are the best spin [1:02:29] doctors in the world. They are not going to define it as a catastrophe. I have to say I have a lot of [1:02:37] conversations about AI nowadays. This is one of the gloomiest I've had. I mean, I was sort of expecting [1:02:42] that. But I guess two very practical questions. One, how much do you yourself use AI? Have you, [1:02:51] you've clearly, you've developed an incredibly sophisticated and alarming sort of perception of [1:02:58] what it could be like. I'm just wondering how much is it part of your actual life or do you watch it? [1:03:04] I'm very careful about it. I get terribly annoyed, like I try to write an email and these AI is [1:03:10] popping, should I help you write your email? No, thank you. I don't want your help. But I don't [1:03:14] know how to shut it down because the companies keep kind of making it more and more intrusive. [1:03:19] I now have this issue again with this kind of AI fake of me that's walking around kind of giving [1:03:25] talks in my name. And I think, do I actually need to watch it? Because people are writing me about what [1:03:32] this AI fake of me have said, and I need to react. So should I watch it? But then I allow it in a way [1:03:40] to take over my time and even my thinking. Because just by watching it, I start interacting with these [1:03:48] kind of ideas from the AI. So this is this is the times in which we are living. Now, again, I tend to [1:03:57] talk about the gloomy side, simply because we have enough people talking about the positive sides, which [1:04:04] are completely real. If AI had no positive potential, nobody would develop it. And the positive potential [1:04:12] is enormous, again, in transport, medicine, education in so many fields. So again, I'm not saying stop all [1:04:19] development of AI. Yes, develop it, but in a safe way, and a bit more slowly. So we enable democratic [1:04:32] accountability to take place, it takes time. And we enable people to adjust and make informed decisions [1:04:40] about which way they want to go. And for that, the most important narrative to combat is the narrative [1:04:47] of inevitability. You wrote in in Nexus, I was very struck by this passage in the prologue, you said, [1:04:55] why are we so good at accumulating more information and power, but far less successful at acquiring wisdom? [1:05:02] And I read that. And I came to mind as I've just been digesting what you've just told me, [1:05:08] it suggests to me that if I if I asked you to put a probability on us having the wisdom to navigate [1:05:15] this, it would be quite low. I don't know. But there is a fair chance. Why? Because we are up against [1:05:22] the wall. Humans often do the wrong things until they have no choice but to do the right thing. And [1:05:31] we are in that position. Because if we choose unwisely in the next few years, we could lose all control of [1:05:41] the world and of the future. And that's the last choice we make. So it's like we had a genie in the [1:05:45] battle. It gave us 100 wishes. We used 99 of them. The world is not in such a good shape. We have one [1:05:52] last wish left. We need to think really hard what to use this wish for. And that's wisdom. So AI, in a way, [1:06:04] could be the catalyst. So for some kind of, you know, spiritual revolution of humanity, simply because [1:06:12] it forces us to think really hard, what does it mean to be human? What is the value of being human? [1:06:19] And the superficial answers that we gave for 1000s of years, oh, humanity is about intelligence, we know [1:06:26] how to play chess, and how to invest money better than the chimpanzees. So this is what makes us so [1:06:32] special. This answer is now irrelevant, because there is something on the planet that knows how to [1:06:40] play chess and make money better than us. So dig deeper, and try to find out so who are you really, [1:06:48] if it's not about playing chess and making money? I think that's a place to end on AI is our last, [1:06:56] we are our last chance. But it could be it could be the thing that forces us to make or enables us [1:07:03] to make wise choices. I'm pretty sure that the world has a lot more wisdom, thanks to your writers. So [1:07:09] Yuval, thank you so much for joining us. Thank you.