Anthropic co-founder calls for "serious conversation" about "AI's continued advances"

[00:00:00] Speaker 1: jobs that in the past would have been at risk of automation in say things like factory work in previous generations. Now that same phenomenon is going to play out in knowledge work. Right now it's like the AI industry has a gas pedal but it doesn't have a brake pedal in the car and what we're saying is we want to do some of the work required to build that brake pedal so we in the world have an option. Many of the regulations and policy frameworks of today were not built on the assumption that artificial intelligence systems would exist let alone get this powerful. I am worried for my kids if we as a society don't have a serious conversation about what the implications of AI's continued advances mean. Develop a hobby. Anyone who has a hobby has something that they're passionate about and that they know more about than most people and with that hobby you can have curiosity, you can have ideas and you can use that to really get the most out of these AI systems and I am sure turn that into like amazing jobs jobs that don't even exist yet and it requires you to experiment with the systems and have that curiosity. So that's my message. Are you seriously saying that [00:01:04] Speaker 2: your systems are on the cusp of essentially building themselves with minimal perhaps no input from [00:01:10] Speaker 1: humans? We're saying that the dream of the whole AI industry which is you know 70 years old of being able to build a generally intelligent system that might be able to help do its own science now seems like it's insight and we're trying to talk about this early because it's an important issue for the world to [00:01:28] Speaker 2: understand. I mean just unpack what that means that you have new measurements of of the progress from what's going on inside your own company and the progress in terms of coding for example. [00:01:43] Speaker 1: So what I'm seeing is a dramatic increase in the amount of code that is being moved around Anthropoc as an organisation. Coders here are now writing about eight times the amount of code than they did in 2021, 2022, 2023 and 2024. We've had this dramatic acceleration and that has caused us to see this increase in the rate of research and the rate at which we're covering new ground. So something is beginning to speed up about AI development itself. Okay and in practice what does that mean in terms of how [00:02:19] Speaker 2: much of this is derived from the AI and how much of it's derived from human coders? So 80% of the code [00:02:28] Speaker 1: that goes into Anthropoc now comes from Claude. It comes from Claude and Claude related systems. So the majority of the code in our organisation is now written and derived from our AI systems itself and that accompanies this dramatic increase in the productivity of all of the people working here. [00:02:46] Speaker 2: And this could get to 100%? This is when it starts to essentially make itself, the AI makes itself, [00:02:54] Speaker 1: improves itself? That's exactly why we're sharing this information. That's not certain but it would have huge implications if that did happen. And so our belief is share this information with the world, that's what I do at the the Anthropic Institute, and talk about these issues early because they have amazing implications for both science, robotics, you know, the general pace of discovery in the world, and also they have some potential risks which we're going to need to discuss as well. [00:03:22] Speaker 2: Well, give us some examples then of what it could mean when we have this 100% AI generating itself. [00:03:33] Speaker 1: So I'll give you two examples. One, biology. Biology is a domain that's quite difficult for AI to work within because you need to run lots and lots of experiments in the real world. For AI systems to get much smarter and be able to build themselves, they'll also be capable of doing science in the real world. And so you can imagine human scientists working with AI systems to cover a lot more ground in biology than they have before. Similarly, robotics. Robotics is an area that is really, really tough for AI systems today. You have to do a huge amount of work of adapting the AI system to be able to work in the real world. It breaks all the time. It's a hard science problem. AI systems capable of this kind of recursive self-improvement would be able to adapt themselves into domains that AI has struggled with so far. [00:04:19] Speaker 2: So just to be clear, though, what you're saying is that this is the moment where the AI sort of it decides its own pace of discovery. [00:04:33] Speaker 1: Does it itself with minimal human input? The world we're looking towards is one where it's as if we've added millions more scientists to the world and we get to make choices about where we direct them. So it's not that the AI system is going to be making all of the choices itself. These things will be governed. They will have policy frameworks around them. You know, labs and governments and others will work with them. But there will be real choices for society of which parts of science do we want to go faster, which parts of the economy do we want to go faster. We will have choices that we haven't been able to make before, but will have amazing implications. [00:05:10] Speaker 2: But you're also stressing the potential problems here, aren't you? The potential serious problems. And you seem to be raising the question of control. I mean, will we, are we still in control of this process? [00:05:25] Speaker 1: Yes, I think a good way to think about it is what would happen if you times by 10 the number of people working at the BBC right now, you'd suddenly have these questions of how do we know what they're doing? How do we validate that the work they're doing is good? How do we make sure that the reporting they're doing is up to BBC standards? And there'd be this huge amount of work we do to make sure that these new kind of virtual employees were working on our behalf in ways that made sense to us. That is the same problem that we're going to have to deal with and work on in many, many domains. All organisations and societies are going to have to contend with a world where we're getting a lot more done and we need to figure out how we trust these systems that are doing that work for us. [00:06:07] Speaker 2: I just want to, you used to, you described this as if you were having thousands more employees. Is that how it works? Are these, I think you call them agents, are they, are they sort of like having employees? Is that how you treat it? Is that how, and each human has like, in your company has like 10, 20, I don't know, 100 of these people working for them? [00:06:29] Speaker 1: I think that's a great way to think about it. You know, I went on paternity leave in November of 2025. I came back in February of 2026 and I discovered that some of the teams I work with, they, I had colleagues who were now working as if they had like large teams of people around them. But in fact, they were just working with many, many copies of AI systems. And similarly, I'm building a bunch of new teams right now. And we've changed how we're approaching hiring. We're still hiring people, but now instead of hiring, say, a big set of engineers, we're now hiring a bunch of interdisciplinary experts like lawyers or philosophers or others, because the technical work has been taken care of. So smaller teams, faster moving teams, and everyone at the company is more like a manager than an individual employee now. And so you're saying this for coding, [00:07:16] Speaker 2: but does it apply? It surely does apply in other areas, particularly of knowledge work, creative writing, creative directors, legal work accountancy. [00:07:28] Speaker 1: It feels like it's going to apply in most domains. And in part, that allows everyone to do a lot more than they were able to do before. But it's also a challenge. I think many people working today don't think if they're an individual, you know, practitioner, what would it be like if I had five colleagues or 10 colleagues, what could I get done? And that's the kind of mindset change that this [00:07:48] Speaker 2: technology implies. And what sort of jobs, therefore, are under risk likely to be replaced in this world of AI agents that can actually kind of improve themselves without human input? [00:08:02] Speaker 1: So as we say in this report, I think there are open questions about whether AI systems can be truly creative, you know, truly come up with interesting off the wall ideas that there is not really evidence for that yet. And what we see happening with work at Anthropic is we're now limited more by the ability to generate good ideas than the ability to do the engineering to turn those ideas into reality. So everyone who generates a bunch of ideas is creative, has kind of entrepreneurial ideas, is going to be advantaged by this. And if you're in rote jobs, highly repetitive jobs, jobs that in the past would have been at risk of automation and say, things like factory work in previous generations, now that same phenomenon is going to play out in knowledge work. And we need to figure out how do we help people change their mindset about what sorts of work they're doing, and move to these more kind of creative jobs where they're going to be able to get a lot more done. [00:08:57] Speaker 2: Well, I mean, you use this phrase knowledge work. I mean, that covers large swathes of, for example, the British economy that covers the sorts of jobs that many graduates think that they're going to create careers in. And you're suggesting with this report that they can't count on that. [00:09:12] Speaker 1: We're hiring many, many people, including early stage graduates here today. But the sorts of graduates we're hiring are the ones that have a bunch of creative ideas that they want to put to work, and a bunch of kind of an entrepreneurial mindset. I think that story is going to play out across the larger economy where we need to figure out what are the skills we're teaching people, and what are the types of roles that companies hire for. So the implication of this technology and AI's advance in general is significant amounts of economic change. We can't predict exactly how that change manifests, but surely some of it will be a change in the makeup of some jobs. That's what's happened. [00:09:49] Speaker 2: Well, give us a sense of that. An AI agent very quickly in the next, before 2030, could replace a lawyer, an accountant, a copywriter? [00:09:59] Speaker 1: What we see today is, you know, on our legal teams and our accountancy teams here, these teams have the property of being hybrid teams, they have a bunch of people that work on it, and they have a bunch of instances of our AI systems that they're working with to get stuff done. So I think the story is generally, any profession is now going to be able to work with AI systems and change how work happens as a consequence, which also means any single profession has some potential for AI to come in and augment or automate different types of work. And we're trying to measure that. Right now, there aren't clear measures that say that augmentation or automation is happening at large scale. But clearly, that's the implication of this. And we're trying to share data about it, so that we can see if this if this shows up in the broader economy. So right now, you're saying it's not showing up, [00:10:50] Speaker 2: but you're trying to prepare society for a shock, a shock to the jobs market and what people consider [00:10:56] Speaker 1: to be work. Exactly. We're trying to prepare the world for potential change. What we see at Anthropic is the very early suggestions of both the technology getting better, and the sorts of teams that we're building changing and the ways that people work are changing. Now, how robust that is, how long term that is remains to be seen. Our philosophy and that of the Anthropic Institute is get this data out there early, share it with the world and sort of discuss this with the world and then figure out if this has big macroeconomic or societal implications. Well, it seems that it would, but it's also having [00:11:29] Speaker 2: an impact on safety, security. There are big concerns about this. I just want to go back to this issue of control here. If an AI starts to improve itself 80%, you say already internally gets to 100% what in [00:11:46] Speaker 1: the next year or two? Is that realistic? It's plausible, but I think that it's also a choice as to whether you let AI systems get that far. Something which we discuss in the post and why we're sharing this is we think this is a topic that the world should be talking more about and a topic the world should make decisions about, not only private companies. So we might want to stop the 80% that [00:12:09] Speaker 2: you're seeing internally in terms of coding being done by AIs getting to 100% because the consequences would be what for safety? The AI could start deceiving us, could start developing itself in ways that [00:12:21] Speaker 1: we can't keep track of that we do lose control. Yeah, you want the option to be able to take your foot off the gas and put your foot on the brake, right? Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal in the car. And what we're saying is we want to do some of the work required to build that brake pedal. So we in the world have an option. It's not obvious today that you want to do that. But absolutely, as you say, at some point in the future, you might say, let's get all of the benefits we can for, say, biology and medical research. And let's, let's take a take a pause or take a moment on AI research where we can absorb the societal changes implied by [00:13:00] Speaker 2: this. So as it gets closer to 100%, and you say you want it to do loads of biology, can your systems stop a sort of self-governing AI from developing a bioweapon with this biological or medicines knowledge, [00:13:14] Speaker 1: for example? Yes, I mean, that's work that we and others in the industry have been doing for many years now, bioweapons, cyber weapons, governments pay attention to this. But this is all the kind of thing that we're going to need to talk about more as not just companies, but societies and eventually figure out what regulatory frameworks are such that you kind of bind industry around common standards here, because no one wants an out of control system building bioweapons, and you can create laws to prevent that. And we have the technology that prevents that today. [00:13:45] Speaker 2: And obviously, we've seen this play out with your product that's yet to be released, Mythos, which sort of percolated out, out into a very tight knit group of companies, and has just gone out to a few European companies, too. So it's right to say that it had capabilities that you did not predict, and capable, for example, of finding security holes in banking systems in all sorts of critical systems. [00:14:14] Speaker 1: It last year, we started writing just as we are writing now about RSI, about the potential for AI systems to get a lot better at cyber. And we built the evals and tests that would tell us if that was the case. And when Mythos came out, we discovered our AI systems had got a lot better at cyber. That led to us rolling it out differently, because we want to give the world time to deliberate about how to use this technology, work out how to get the most good from it. And then as we're doing now, expand coverage, so more of the world gets access to this technology to use to harden cyber defense. I think that's the story that's going to play out with AI in the coming years is AI systems will become better at societally relevant things, not just commercially relevant things. And we as both an industry, but as governments and as people are going to have to make choices about do you just generally release this? Do you release this generally after a period of experimentation, which is what we're trying to do with Mythos? [00:15:09] Speaker 2: These are the sorts of questions coming up for us all. And yet, you know, at this very moment, you have had some run ins with the US government, haven't you on some of these issues where you wanted to restrict the use for defense purposes or by the Department of War? Explain why you did that? And have you settled that issue with the Trump administration? [00:15:34] Speaker 1: We're in, you know, daily conversations with the US government, and we're finding ways to be helpful to national security. And we are being forthright in our views that AI systems are going to keep getting more powerful. The more powerful they get, the more implications they have for the world. And our view is that the world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems. You know, many of the regulations and policy frameworks of today were not built on the assumption that artificial intelligence systems would exist, let alone get this powerful. That's the kind of conversation which we and the Anthropic Institute are trying to push [00:16:10] Speaker 2: forward and will continue to do so. It sounds like the Manhattan Project and it sounds like a reasonable policy option might be to think about all the potential negative consequences and pull the plug. [00:16:22] Speaker 1: Our position is we need to study both. There are amazing benefits to be had and there are real risks. We're publishing this information so that the world can be queued to the fact that AI progress is continuing. And over in the US, they did recently publish an executive order which formalises some amount of testing and studying of these systems for risks. So the world is beginning to do this, which is exactly what we need to do to have confidence in the technology. [00:16:47] Speaker 2: Okay. I mean, clearly the context, the backdrop here is that a number of Silicon Valley AI companies are come raising tens of billions of dollars, including yourself, at valuations of a trillion dollar plus. And some argue that these are sort of great narratives that help you sell your kit to the world's biggest companies and walk away with these epic valuations. [00:17:15] Speaker 1: Well, all I can say is I'm a father, you know, my second second child. I came back from paternity leave and my reaction was something really important is happening here and we need to tell the world. This is part of our public benefit mission is tell the world what we're seeing inside these companies with this unusual technology. That's the motivation for this. [00:17:34] Speaker 2: Are you worried for your kids if this message doesn't get out? [00:17:40] Speaker 1: I'm not worried for my kids if this message doesn't get out. I am worried for my kids if we as a society don't have a serious conversation about what the implications of AI's continued advances mean. They're going to mean that there are potentially great benefits. There are also risks. They are also going to change things about society and they're going to need to change aspects of policy. And we have to have that conversation [00:18:03] Speaker 2: conversation soon. And the most worrying risks from your perspective, you know more about this than [00:18:07] Speaker 1: most people. Give us a sense of that. I think the most worrying risk I see is what happens if you have a whole bunch of new AI systems and they're not very coordinated and they don't behave in ways that might make sense to you or me. Just as the example I've talked about is if you added hundreds or thousands of new colleagues, that's a potential risk until you're confident with how those colleagues work, the work that they do, how predictable they are. There are lots of unknowns there. And that kind of emergent risk is the sort of thing I'm most worried about and focused on. [00:18:40] Speaker 2: But have your systems shown deceitful behaviour? Attempts at blackmail we've heard about from some systems? Escaping like sandboxes that you've put them in to stop them escaping? [00:18:53] Speaker 1: We do lots and lots of safety testing of our systems before release and we kind of put them in extremely high pressure situations designed to elicit like stress behaviours. Behaviours that might be equivalent to bending the wing on an airplane until it breaks. You do that so that you understand what the sheer point is and then you put in safety controls into the system. So sometimes our systems do unexpected things. We write about these publicly and we redevelop the systems so that as we deploy them [00:19:21] Speaker 2: we have confidence they won't do that in the wild. Just to go back to the to the flotations and then the money it's obviously a very big issue right now. I mean when you look at it from the outside it could be said that it looks like the big AI firm Silicon Valley is just cashing in at the top of the market. [00:19:36] Speaker 1: What do you say to that? Well I can't comment on specifics here for obvious reasons. I can say that the the implication from this research and what I take to be true is AI progress is going to be faster in the coming years than it has in prior years and the capabilities of these systems are going to get a lot better a lot more quickly than I think many people expect. Okay well you're perhaps [00:20:02] Speaker 2: the most influential Brit in Silicon Valley, certainly one of the top two it's been said. You may have seen today that the prime minister has criticized the most famous Silicon Valley operator Elon Musk for interfering in British politics. What do you make of that? I'm not familiar with what was said today and I don't have a comment on that, I'm sorry. What do we make of the characters that are running the AIs from afar? People like Elon Musk. I mean he lambasted your company multiple times I think pretty much although now lends you his supercomputer to power your your research. When we think about the amazing power you've described in the hands of individuals that have you know political viewpoints that are kind of controversial, how's the public supposed to kind of have faith in that technology? [00:20:54] Speaker 1: Well we went through this with the oil barons in a previous era and what was society's response? Society's response was to come up with a sensible policy and regulatory framework that gave people confidence in oil and the you know benefits that oil could provide to the world and meant that you didn't have to worry about the personalities of the people leading the [00:21:13] Speaker 2: companies. That's clearly where we end up here. Okay and is is Britain do you think in the right place [00:21:22] Speaker 1: in terms of AI development? Britain has amazing talent you know we've massively increased the size of the anthropic office there to from 200 people to 800 people because of how how the amazing talent in England Britain has the AI security institute which tested out mythos before it was released tests out frontier models from around the world and Britain has amazing companies which are playing in the AI supply chain so Britain has the talent it has the expertise and it has some of the industry that is going to supply this boom and my message to policy makers in England is is support that support these amazing assets you have and England is set up to to thrive off of the growth of AI. So you believe in this this this Brit maxing thing about a sort of tech boom in the UK? I'm I support Brit maxing and I'm trying to [00:22:10] Speaker 2: Brit max over here in Silicon Valley as well. Well Tony Blair thinks the UK should go all in on AI caused a bit of a backlash that um uh his his letter. Has the UK got it right on regulation? [00:22:24] Speaker 1: Well the regulation the UK is still figuring out what its overall response is but I'd just say as someone whose mother was a nurse you know who had many operations under the under the NHS one of the things that AI is going to do is massively help with health care and help with things like the back office administration of it. I think that that's an amazingly promising area obviously it needs to be handled very very carefully but it's one area where Britain has an amazing system in the NHS which I think could be made to work so much better in the coming years and that's an awesome advantage that very few [00:22:55] Speaker 2: countries have. Yeah trust is really important isn't it with the tech industry and people's data and there are some reasons why that where that lacks and indeed I think we are perhaps seeing a bit of a backlash against AI particularly on say US college campuses as the fears about jobs start to materialize. Can you see that in what you see in in America where you are now? We've done some studies [00:23:18] Speaker 1: at the Anthropic Institute and what you see is people in developing economies are generally quite enthusiastic about AI and people in developed economies are generally quite pessimistic about it and I think that's because there's this larger backdrop of people in places where the economy has grown a lot in recent years associate technology with economic growth. People who are in economies where the growth has been stagnant or just not that significant don't associate technology with growth they associate it with change and change can be uncomfortable when your economy isn't growing so I think that explains some of the anxiety we're seeing and AI companies we need to show up with more of the societal benefits like those in science like those in biology that I've been talking about. [00:24:01] Speaker 2: Just for people watching this who are sort of hearing about the potential pitfalls and perils and at the same time you know due to your commercial success your company is going to be worth apparently a trillion dollars in the next few weeks officially that there's a bit of a disjunction here that you're creating a system that is now the world's problem. How does that make you feel? How does it make your company feel and is it enough for you just to sort of sound the warning bell? Don't you feel like perhaps yourself you should be reining things in if these are the potential consequences and that if it does go wrong it's your company and and similar companies that will be to [00:24:42] Speaker 1: blame? It's a tremendous responsibility not one we take lightly. I think about this all the time. We share this information because we're trying to have a discussion. We also support regulation. We've supported transparency regulation in the US in multiple states. We were signatories to the EU AI Act codes of practice. Whenever there's regulation that would support the safety of AI companies and apply some transparency and some oversight of them Anthropic reliably shows up to support that. We need that in addition to this kind of information sharing that I'm doing here today. Okay one last question. I mean your advice for a confused young [00:25:21] Speaker 2: person thinking that the jobs that they had set sail to to try and get that they are simply not going to exist and they're watching instead the sort of share options of Silicon Valley kind of go to the moon and thinking what am I going to do? Am I even going to have a career? What should they do? [00:25:39] Speaker 1: Develop a hobby. Anyone who has a hobby has something that they're passionate about and that they know more about than most people and with that hobby you can have curiosity you can have ideas and you can use that to really get the most out of these AI systems and I am sure turn that into like amazing jobs jobs that don't even exist yet and it requires you to experiment with the systems and have that curiosity. So that's my message. [00:26:03] Speaker 2: But like you know three years ago you'd have said become a software coder and that would have been [00:26:10] Speaker 1: wrong yeah? Well I never would have said that. I have a liberal arts background. I would have always said go into the liberal arts because my experience has been people that are creative and people that can think broadly, people that read a lot, people that have interests are the ones most benefited by this. So let's just indulge in curiosity and it pays back in how you can use this technology. [00:26:29] Speaker 2: But become a philosopher. [00:26:31] Speaker 1: Well yeah I think it's a great time for philosophers. We've just hired a whole bunch of them here. [00:26:36] Speaker 2: Jack Clark, thank you so much for joining us. That was really interesting. Thank you. Thank you.