Inside Anthropic, the $965 Billion AI Juggernaut — The Circuit

[00:00:00] Speaker 1: I love this library. [00:00:05] Speaker 2: It is an absolutely beautiful library. [00:00:08] Speaker 1: Are you a big reader? [00:00:10] Speaker 2: You know, I mean, I read a lot in general. I don't know that I've had that much time over the last year or so. [00:00:18] Speaker 1: Dario Amadei is an unlikely AI celebrity. He's known for warning the world about the risks of artificial intelligence. Now, his company is an AI frontrunner, valued at nearly a trillion dollars. You're shipping so much, so fast. How are you doing that? [00:00:34] Speaker 2: We use Quad, you know, across the product development cycle, and it allows us to release very fast. [00:00:40] Speaker 1: Founded by a team of OpenAI defectors in 2021, Anthropic started out as an underdog lab. Today, it's the breakout star, wiping billions in value off software stocks, going head-to-head with the Pentagon, and creating models allegedly powerful enough to burst through the walls of modern cybersecurity. [00:00:59] Speaker 2: Some of the early companies that we gave this to said things like, this is a superweapon. Please don't release this. [00:01:09] Speaker 1: Guiding Anthropic through this is a sibling duo. Dario, the brother and visionary, and Daniela, the sister and operator, who puts Dario's swirling cosmic thoughts into action. [00:01:20] Speaker 3: Dario and I, we've always been really close since we were little, but I think we always really wanted to do something big together. Okay, but when you argue, who wins? [00:01:28] Speaker 1: No one. No one. As the AI arms race escalates, the Amadeis are eager to establish themselves as the good guys. But Anthropic's technology could have profound implications for how humanity works, learns, thinks, even fights wars. [00:01:43] Speaker 4: Anthropic is run by an ideological lunatic. [00:01:45] Speaker 2: This is a debate about what the proper use of AI by the government is. [00:01:51] Speaker 5: At some point, the models become very dangerous. You know, the bad guys will have it too. And at that point, we have to make sure that the good guys have an even better model. [00:01:59] Speaker 1: The name Anthropic actually comes from the Greek word for human, which speaks to their mission to build responsible AI for the long-term benefit of humanity. Can you actually do that when you're building the most powerful technology in the world? You are at the center of the AI universe right now. What does that feel like? [00:02:24] Speaker 2: The experience I've had for my whole career, and certainly the whole time at Anthropic, is that there's this kind of smooth exponential. And the experience of the smooth exponential is, nothing's happening, nothing's happening, nothing's happening. Little things happen, and then Zoom, it goes crazy. So, you know, I was watching this graph for a while, and I said, oh, yeah, we'll probably become the, you know, the AI company with, you know, the most revenue and the most valuations sometime around this time. And indeed, it has happened. We're just keeping in mind all the things we usually keep in mind. You know, how do we train good models? How do we put them in good products? How do we make sure that everything's safe? [00:03:04] Speaker 1: What were you like as a kid growing up in San Francisco? I know your dad was a leather craftsman. Your mom worked in libraries. How did that shape you? [00:03:12] Speaker 2: You know, the whole, you know, like, first, you know, internet revolution was happening around me, and I had absolutely no interest in it. I was just interested in, like, doing math and, like, scrawling things. I was interested in, like, you know, understanding the universe. I was interested in science fiction. I think I just felt a lot of curiosity about the world. [00:03:35] Speaker 3: What was it like growing up with Dario? He was so smart. He was taking, you know, calculus when he was, you know, in, I think, like, middle school. He took math classes at UC Berkeley when he was in high school. I was actually more into reading and arts. So we were almost like complete complements in that way. [00:03:55] Speaker 1: Dario studied neuroscience before turning to AI at Baidu and later Google, while Daniela started out as an early employee at Stripe. They lived together in San Francisco, along with Daniela's husband, Holden Karnofsky. Then, in 2016, Dario joined the newly formed OpenAI, followed by Daniela in 2018. The company began as a non-profit that promised a safe, more open path to superintelligence. [00:04:21] Speaker 6: I think we need to think right now about how we want this deployed, how everyone gets the benefit from it, how we're going to govern it, how we're going to make it safe and sort of good for humanity. [00:04:31] Speaker 1: At OpenAI, Dario developed the concept of scaling laws, predicting that large language models would improve simply by adding more data and computing power, even if the underlying algorithm stayed the same. [00:04:43] Speaker 3: At that point in time, I know it sounds crazy now looking back, not a lot of people believed, hey, scale-up is the way that these models are going to get smarter and better. That was sort of an unusual, counter-cultural, scientific perspective that I think was really held by our founding research team. [00:04:59] Speaker 1: That approach helped supercharge OpenAI's models, paving the way for ChatGPT. But the Amadeys reportedly clashed with Altman over the company's direction and values. Altman has said, For all the differences I have with Anthropic, I mostly trust them as a company. Your decision to leave OpenAI has become Silicon Valley lore. What did you disagree on? [00:05:22] Speaker 2: Look, I'm going to say it very simply. There are many valid disagreements to be had on safety. We certainly had some of those disagreements with them, but that alone is not sufficient to leave. When you feel that you can't trust someone, when you feel that their values are not what they say they are, when you feel that they're not honest, that makes it very hard to, you know, to continue to work with the company, to continue to trust the company. And look, at the end of the day, why argue with someone when you don't have the same vision and you don't trust them, the way to resolve it is you go off and do your thing, they go off and do their thing. [00:06:08] Speaker 1: So this is Prasida Park. When Dario and Daniela decided to leave OpenAI to start Anthropic, this is where the early team would get together. It was during the pandemic, so the story goes, and a bunch of early employees would come here, they'd pull up a chair on the grass, they'd have lunch, and they'd talk about what they were building. [00:06:30] Speaker 2: When we started this company, there were seven co-founders, of which me and Daniela are too, and now, you know, we're basically the only company in the space that has all of its co-founders still here. You don't get to, you know, be a company of the size and scale that we are with that happening. Like, that almost never happens. [00:06:48] Speaker 1: From the start, Anthropic pitched itself as the ultimate safety-conscious AI company. Dario has published long essays with names like Machines of Loving Grace and the Adolescence of Technology, musing on the miraculous potential of AI, as well as the worst-case scenarios. Anthropic's chatbot, Claude, has been trained to follow a set of principles called a constitution, intended to keep it on the straight and narrow. Claude has a very distinct style and feel, a human name. What are you trying to convey? [00:07:20] Speaker 3: I think when people interact with Claude versus, you know, other systems, there is more of a feeling of, I like to describe it as professional warmth. So the goal is not for it to be your best friend, but it's not for it to be sort of cold, rote, calculating. It should feel approachable, but distant, right? Professional. [00:07:37] Speaker 1: Anthropic has talked about teaching Claude to be good. What is a good model? What is a bad model? [00:07:44] Speaker 3: You don't want a model that lies accidentally or intentionally, right? Lying we call hallucinations. It makes something up. The models are just trained to predict the next word, so sometimes they don't know, and they just invent something. Models sometimes, as we've shown in our research, can purposely try to deceive you. We have to make sure that doesn't happen in production models that we expose to customers. And then there's a lot of work around harmlessness, just making sure that the model is not accidentally producing, you know, information that is wrong or harmful or could cause someone to do something bad. [00:08:13] Speaker 1: Whose values are being put into Claude? Like, is there a universal good? There's so many religions, so many different beliefs. Like, how do you even decide that? [00:08:21] Speaker 3: So, of course, there's no universal standard for what makes something helpful or harmless, but there are founding documents in human history, like the UN Declaration of Human Rights, that we can use to train Claude's character. And I think, you know, interestingly, on the religious front, we've actually started to have a lot of conversations with religious leaders around how should we think about, you know, Claude, the entity, and how can we bake in some of the kind of core values that are consistent across religions, types of belief, that really just transcend, you know, a specific worldview, but that are really things that human beings have been grappling with for millennia. [00:08:57] Speaker 1: Have you tried to give Claude a trait that didn't take, or was there any trait that emerged that surprised you? [00:09:03] Speaker 3: I think in the early days, it's funny to look back on them now, the kind of early Claude's, Claude II era, sometimes Claude would be almost a little bit nannyish. Claude was like, I'm really concerned about you. And you're like, Claude, I was asking for the weather, right? It was something really benign. Thankfully, we didn't, you know, release most of the most egregious versions of those. But I think it is, you know, it is, when you think about it, it's like tuning a dial. And so our researchers have this very fine needle to thread in order to land at the center of that. [00:09:31] Speaker 1: Whatever Anthropik is doing with Claude, it seems to be working. Anthropik's revenue has skyrocketed over the past year, making the company profitable for the first time. That's largely thanks to the company's focus on more lucrative business tools. Claude Code, a major leap that automated large chunks of software engineering, and Claude Cowork, which gave that power to everyone else. Now, early on, others focused on fun, splashy consumer apps. You made a bet on coding and enterprise. Why did you make that bet? Was it a values decision or a business decision? [00:10:07] Speaker 2: Look, if you pick a business model that fundamentally conflicts with your values, you're going to have a hard time, right? Either you betray your own values or you become irrelevant. And so when we thought about it, we said, look, you know, we've seen the world of social media, the consumer world, it really seems to, you know, encourage engagement, even addiction. You know, the slop we've seen with AI video models, it's like, what's going on? Does it want to maximize the number of minutes that you're paying attention to? Because that's the advertising revenue-driven incentive. Whereas if we look at enterprise, look, I mean, you know, we want to make these models useful to people. We want to use AI to, you know, cure diseases that we couldn't cure before, right? Well, that's working with biotech, it's working with pharma, it's working with academic research groups. All of those are enterprises, right? We want to use AI to, like, you know, to make energy cheaper and more efficient. That's all enterprise. And so I think it served us well to have this business model that largely aligns with our values. [00:11:10] Speaker 1: Soon after Claude Cowork was released, $285 billion in market value vanished overnight. Traders called it the sasspocalypse. [00:11:18] Speaker 7: This kind of white-collar wipeout story in the software sector, terrifying. [00:11:23] Speaker 1: Some of those are down for nine days in a row, so clearly the tension is building. If AI continues improving at this pace, how much of traditional software gets replaced and how fast? [00:11:34] Speaker 2: I think with AI, like, the pie is getting bigger, right? So the existing incumbents may be smaller in relative terms. Some of them may go down in value. Some of them may even go out of business if they don't adapt in the right way. But, like, I would guess that the software industry gets larger, not smaller, although there will be some big losers. Those who don't kind of see what's coming, who don't identify the moats they have, they're going to have a really hard time. [00:12:02] Speaker 1: Anthropic's recent growth spurt might not have happened without this man, Boris Cherny, the engineer behind Claude Code and Claude Cowork. When Anthropic hired him in 2024, Cherny was living a very different life in rural Japan. [00:12:17] Speaker 5: There was a lot of farmer's markets, like, very slow. We were making miso. That was sort of the big hobby. And I remember using the first AI chatbot that I'd ever used and it just took my breath away. And I was just like, oh my God, I have to just be a part of this. And I'm also just such a big sci-fi reader and so I just know how bad this thing can go. Like, this technology is incredibly powerful. And so we moved back. [00:12:38] Speaker 1: You created Claude Code. You led the development of Claude Cowork. What problem were you trying to solve? [00:12:47] Speaker 5: If you look at the coding products, they were all pretty simple. It was like, it was sort of like, you know, like complete the word, complete the sentence. That was the extent of AI in coding. And we just wanted to make a way bigger bet. And our bet was we think actually a coding agent can do all of it. A year and a half ago, you wrote the code by hand. And sometimes you press tab and it would autocomplete a line. Now I talk to my Claude and it writes the code. And then while it does that, I talk to the next Claude and it writes some code. And at any point, I have either, you know, like a few Claude's running and up to a few thousand Claude's running, doing things. [00:13:21] Speaker 1: How much code is Claude writing internally for engineers here? [00:13:26] Speaker 5: It's writing almost all of it. On my team, so for me personally, it's been writing 100% of my code for at least six months. The work of engineering has just completely changed. I feel like I suddenly have superpowers. I have like a jetpack and engineering has never been this fun. [00:13:41] Speaker 1: Can we give Claude a spin? [00:13:43] Speaker 5: Yeah, let's do this. [00:13:44] Speaker 1: All right. [00:13:44] Speaker 5: All right, so let's make like a little recipe app. So make a recipe app. What do you want it to do? [00:13:50] Speaker 1: I would love it to suggest meals for the week. So, you know, [00:13:56] Speaker 5: this is going to take maybe a few minutes and let's just see if Claude builds this. [00:14:01] Speaker 1: All right. Oh, it gave me some options. Hmm. Let's do... [00:14:05] Speaker 5: These look pretty tasty, actually. [00:14:07] Speaker 1: I could do a Greek chicken power bowl. When I click on a recipe, I want it to show me how to do it. Thank you. Aw. [00:14:21] Speaker 5: I'm sure Claude appreciates it. [00:14:23] Speaker 1: I always try to say thank you. [00:14:25] Speaker 5: Yeah, I always try to be nice, too. I don't know. [00:14:27] Speaker 1: A recipe app might not be the most mind-blowing demo of Anthropix technology, but what Claude Code just did in minutes used to take hours or days. For some, that looks like a big opportunity. Are you, like, nervous about next week or is it, like... What's next week? Your developer conference. [00:14:43] Speaker 2: Oh, yes, yes. There's so many things happening every week. I know. [00:14:51] Speaker 8: Year over year, API volume is up nearly 17x on the Claude. In the last 12 months, we shipped eight frontier models to developers and users. [00:15:01] Speaker 1: Welcome to the second annual Code with Claude conference, where Claude superfans come hoping to get a glimpse of the future. We're having a lot of fun. [00:15:09] Speaker 3: There's a ton of adrenaline. [00:15:11] Speaker 2: This is the first year we've grown faster than the exponential. In the first quarter of this year, we saw, if you were to annualize it, 80x growth per year. Ooh. [00:15:25] Speaker 9: I use Claude every day in the form of Claude Code to do all sorts of stuff, not just write code. [00:15:31] Speaker 7: I write way more code than I did five years ago. I've got the confidence of, I don't know, a 22-year-old with VC money. I'm not 22. [00:15:40] Speaker 10: My friend invited me, and it's been as amazing as someone that's not really technical, which I think is really interesting just to see what everyone's working on, what's out there. [00:15:49] Speaker 9: I think the concerning part and the exciting part is, hey, they can do things that we couldn't dream of and timelines we couldn't ever expect. But the kind of alternate side of that reality is you really have to be prepared because they will really be 1,000 times more productive. [00:16:07] Speaker 1: All of this raises an obvious question. Will engineers be the first casualties of the AI they're building? It's been revenge of the nerds for a while. Is that over? [00:16:19] Speaker 5: I think we all become nerds. [00:16:21] Speaker 1: But what happens to the actual nerds? [00:16:23] Speaker 5: I think the actual nerds, well, they have to figure it out. Yeah, I think for a lot of them, the skill that they had before is going to help them in the future because they sort of have a pretty big head start. They do a lot of things that are not coding. Engineers also have to talk to users. They have to plan. They have to think about what's next. So I think these are still parts of engineering that are going to stick around. [00:16:48] Speaker 1: Silicon Valley may have AI fever, but elsewhere, the mood is less upbeat. 70% of Americans think AI will kill jobs, and nearly a third worry, theirs will be one of them. Dario Amede has been outspoken about this issue, and some of his predictions don't exactly sound reassuring. [00:17:06] Speaker 2: I think we could have this very unusual combination of very fast GDP growth and high unemployment or at least underemployment or, you know, low-wage job, a lot of low-wage jobs, high inequality. [00:17:21] Speaker 1: You've been really direct about job loss. AI could eliminate half of all entry-level white-collar jobs in the next one to five years. That was a year ago. AI has moved incredibly fast. Is it still 50% or is it higher? [00:17:35] Speaker 2: I don't know exactly, but I'm still pretty concerned. I'm still the same order of concerns. You know, we are seeing right now that AI is making people more productive, but that's the usual hump. You automate 90% of the job, great, people are 10 times more productive in the other 10% because they're 10 times more leveraged, but eventually it gets close to 100%. Now, the sequel to that is, well, then you have to find something else for them to do. Right now, AI makes the software engineers more productive, even though AI writes all the code or almost all the code. But we're already starting to see the beginning of like, you know, there may be some people that it's not making more productive, that it's better for the AI to just do the thing. [00:18:15] Speaker 1: How does that sit with you? [00:18:17] Speaker 5: It's very uncomfortable. I think this is the reason why I chose to go to Anthropik. And I think this is the reason that a lot of people here chose to go here, is artificial intelligence is this force that is far bigger than we are. But here, we can hopefully make it go a little bit better. [00:18:36] Speaker 1: Do you feel like it's your job to do something about that? Or is that someone else's problem? [00:18:42] Speaker 5: I think it's a thing that we have to talk about. We have to advocate for it. Ultimately, it's up to society to solve it. This is bigger than one company. [00:18:49] Speaker 1: There has been a lot of pushback on, you know, and I know you've said you're trying to warn people, but that, you know, Jensen Huang said you're conflating tasks with jobs. [00:18:58] Speaker 9: AI is creating jobs. Anybody who is saying that AI is wiping out jobs is scaring people. [00:19:04] Speaker 1: Other folks have said this, you know, it's sort of doom marketing. That benefits Anthropik. [00:19:08] Speaker 2: So I want to be really clear and push back hard against this. In every interview, I talk about the possible ways to address these risks from tax and macroeconomic policy to what the new jobs are. In the adolescence of technology, I have like five pages where I lay out the difference between tasks and jobs, why this time is different than other times. But social media, which I detest, which I detest as a category, people have these three-second clips risks from a year ago. I've written much more carefully about these things where I talk about the risks. So these are, the idea that this is cheap marketing is itself cheap marketing. I think it's part of the disease of Silicon Valley. It's been caught up in this social media world of three seconds. And so my message is just definitely not doom is coming. My message is like, this is something that we should see coming, that we're worried about, and that we need to actually respond to positively. [00:20:13] Speaker 1: Beyond the software industry, the potential impact of AI on jobs seems harder to predict. Anthropic has published a paper estimating which fields could make the most use of AI in the near future. If its predictions are right, management, finance, and legal jobs could soon look very different. Which jobs go away? Who gets replaced? And what new jobs are created? [00:20:36] Speaker 2: So no one knows for sure. Because, you know, the economy is unpredictable. The thing that we have going for us here is the pie is going to expand a lot. And so because the pie is going to expand a lot, there are probably going to be places where people can go. It's just a matter of finding them fast enough. [00:20:54] Speaker 1: So play this out for me a little bit. You know, you wake up in five years, you know, what does this country look like? What are those people doing? Because if there's that much unemployment, is that not how revolutions start? [00:21:08] Speaker 2: Yeah, no, this is the outcome we want to prevent. This is absolutely the outcome we want to prevent. I think there's a few places. None of them are guaranteed, we're not sure, but there's the physical world. We need a lot of more people to make, build, manufacture things in the physical world. Anything that's human-centered, I think that's going to be a big deal, right? People, or at least some people, want to talk to humans. So these kind of human-relationship-driven jobs, like, I think those are going to be important. And I think there'll be some effort by the humans to kind of direct the AIs, right? At some level, it has to be in line with someone's values and someone's intentions. And so I think there's going to be some role there, although I don't know how thin versus how thick it will be. [00:21:51] Speaker 3: I think I feel a little bit more hopeful that humans will continue to find ways to leverage AI, to be productive, to do the parts of the work that are meaningful to us that only humans can do. I think the human-to-human interaction will never fully go away. The example I often reach for is what will happen in medicine. Today, we hire doctors who are expert diagnosticians. I think AI is going to soon be pretty good at telling you what the suite of options of things that are wrong with you and what tests to run, and you won't need a doctor to do that. But an AI can't physically examine you and say, hey, does it hurt when I press here? They can't have a bedside manner with you that says, tell me how you're feeling about this. How are you coping with going through this process? And I think we're going to pivot something like medicine to be much more focused on the interpersonal because the diagnostic tools are going to become much better. But the interpersonal human part, that's not going to change. [00:22:49] Speaker 1: Explain this to me. Daniela runs day-to-day operations. All the leadership team reports to you. Yes. No one reports to you. That sounds like a pretty sweet job. [00:22:57] Speaker 2: It's incredibly freeing. It lets me do all the things that I do much more easily than I would otherwise. [00:23:05] Speaker 1: And she does all the work? Is that what you're saying? It's a... I'm just... [00:23:08] Speaker 2: If you... If you had to go through the things I had to go through during D.O.W. or... No. [00:23:19] Speaker 1: The Amadei's hope AI can have a utopian influence on society. But there's no denying the technology's destructive and dystopian potential. Anthropic is at the center of a technological arms race with powerful governments vying for control. Dario has charged head-on into this arena. He's not afraid to blast his competitors. [00:23:39] Speaker 2: And then I think there are some players who, you know, who are YOLOing, who pull the wrist dial too far, and I'm very concerned. Who is YOLOing? So, that's a question I'm not going to answer. [00:23:51] Speaker 1: Or critique the U.S. government, Ananthropic's own partners, for selling AI chips to China. [00:23:56] Speaker 2: It's a bit like, you know, I don't know, like selling, selling, you know, nuclear weapons to North Korea. I've been very outspoken about the need for export controls on chips to China. I say this because I think it would be really bad for America, for, you know, the state of democracy in the world, for, you know, China to be ahead in AI capabilities. And, you know, it's like some of the chip makers obviously don't agree with that view. But it hasn't stopped me from saying it. Even after we've signed more partnerships, I'm sure they wish we didn't say these things, but these things are what I believe. And so, you know, look, we're all adults here. We can work together on one thing while disagreeing about another thing. [00:24:38] Speaker 1: Amadeh's statements about geopolitics date back to his days at Caltech. As a sophomore physics student, he developed a reputation as an anti-war advocate who believed scientists shouldn't sit in ivory towers. It's a worldview that would later define Anthropik's position on the future of war. You've had a longstanding anti-war stance, and yet you were one of the first AI companies to sign a contract with the Department of Defense to operate on classified networks. These are the networks that the U.S. uses to fight wars. Explain that. [00:25:09] Speaker 2: Look, I mean, the world changes. Like, you know, my view of this technology, you know, when I see Russia invading Ukraine, when I see the risk of China invading Taiwan, you know, it worries me that we have a kind of resurgent authoritarian bloc, that they're very aggressive and that we need to defend ourselves. You know, I may not agree with every policy of either administration, but, you know, that's why we've generally been supportive of this. [00:25:36] Speaker 1: You've been working with Palantir since 2024. [00:25:39] Speaker 2: That's right. [00:25:39] Speaker 1: You know, their technology is used by ICE, police departments in Gaza. Is Claude being used for surveillance in other ways? [00:25:46] Speaker 2: We don't work with ICE either through Palantir or anyone else. We don't work with CBP. I don't believe we work in Gaza. We're very careful about, you know, scoping our engagements to things that we believe in. [00:25:59] Speaker 1: In 2025, Anthropic, along with OpenAI, XAI, and Google, won a $200 million contract with the Pentagon. Anthropic framed it as an opportunity to become the leading AI vendor for the government. Claude was reportedly used by the U.S. military in the operation to seize Venezuelan President Nicolas Maduro. Weeks later, everything started to unravel. [00:26:23] Speaker 4: We bring you breaking news from Anthropic. The Department of Defense demand that Anthropic allow full use of its AI technology without guardrails. [00:26:31] Speaker 1: Anthropic drew red lines, refusing to let Claude be used for mass surveillance and autonomous weapons, putting the company on a collision course with the Pentagon. [00:26:39] Speaker 7: The tech giant facing a deadline today to accept conditions or be blacklisted. [00:26:44] Speaker 11: Does a commercial entity have the right to determine how their products are used by the military? Lawsuits were filed, [00:26:52] Speaker 1: and Anthropic was banned from the Pentagon, underscored by strong words from President Trump and Defense Secretary Pete Hegseth. [00:26:59] Speaker 4: Anthropic is run by an ideological lunatic who shouldn't have [00:27:02] Speaker 1: a sole decision-making [00:27:06] Speaker 4: of what we do. [00:27:07] Speaker 1: Do you mind being called an ideological lunatic or a bunch of left-wing nutjobs? [00:27:11] Speaker 2: You know, I've been called worse things than that all the time. [00:27:14] Speaker 1: What does winning this fight actually look like? [00:27:17] Speaker 2: I won't even call it a fight. This is more a debate about what the proper use of AI by the government is. And AI is an emerging new technology. We don't understand the ways in which it's reliable or unreliable. We don't understand the ways in which it promotes our values or undermines our values. And so one of the things that I thought was important was to establish a precedent on some of the use cases we think are good, which frankly is most of them, and some of the use cases that we're concerned about. [00:27:49] Speaker 1: A U.S. official has said with the help of LLMs, the U.S. military has gone from being able to hit 1,000 targets a day to 5,000 targets a day. That means Claude, can help kill more people more quickly. Are you comfortable with that? [00:28:04] Speaker 2: Basically, you're asking, like, you know, do you believe in this country, right? Do you want this country to be a more powerful actor rather than a less powerful actor on the world stage? I do. I'm a patriot. It's not up to me. If we provide a technology, it's not up to us to say, you can do this military operation and you can't do that military operation. Now, I might privately believe that this military operation makes sense and that military operation is a bad idea, but we're not going to deny the technology. You basically have to, you know, you have to leave policy in the hands of the military decision makers. [00:28:38] Speaker 1: Bloomberg has reported that Claude is being used by the U.S. military in the war in Iran to do AI-assisted targeting via a platform made by Palantir MavenSmart system. In February, a U.S. missile reportedly hit a girl's school in Iran, killing more than 150 people, most of them children. Did Claude play a role in that strike? [00:29:00] Speaker 2: We don't know exactly how, you know, these models were used. You know, obviously, like, you know, these things that, you know, mistakes that happen in warfare are really, really terrible. like, this is a really terrible thing to happen. We were willing to risk the future of our company to, like, limit how, you know, these models are used. And, you know, what you're talking about is a use case that doesn't even violate our red lines. We're worried that there will be a hundred times as much, you know, with use cases that do violate our red lines. Now, you know, again, I would say, I think overall, the use of these, the use of these models is appropriate. I think it's good on net. You know, but military decision makers make terrible mistakes, even at the best of times. And I don't know if we're in the best of times. What we've seen here is Claude assists, but a human makes the final call. So a human made that final call, not Claude. Imagine if you had a world in which, not Claude, because we haven't allowed it, but someone else's AI model, the AI model just makes the decision and the human never sees it. That's what we were standing up for. That's what we were fighting against. [00:30:10] Speaker 1: This school had a website. You could have found it in a Google search. Like, shouldn't Claude have spotted that? And does it speak to a scarier issue about using technology as a shortcut in war? [00:30:22] Speaker 2: The principle that was obeyed here is a human makes the, a human makes the final decision. I don't know what role Claude or any other AI had, but like, if this isn't an illustration why that principle is so important, I don't know what is. [00:30:36] Speaker 1: Is AI warfare more likely to stop World War III, a war between the U.S. and China, or is it more likely to make it happen? [00:30:48] Speaker 2: I would say, on balance, it is more likely to stop it. But if we have no limits on how it's used, then I think, you know, it could be more likely to cause it. You've seen Dr. Strangelove, right? The premise of it was like, you know, you have a doomsday device that automatically fires nuclear weapons when it thinks nuclear weapons are being fired at it. What could go wrong, right? I think the way conflicts happen is that, you know, the two sides jump at each other. They misunderstand each other. And when we don't have proper oversight of this technology, I think those kinds of accidents are more likely to happen. Now, I think if AI is used in an appropriate way in not even warfare, but think of just intelligence collection, you know, let's say we're able to, you know, predict an invasion of Taiwan or a new movement in Ukraine, like, you know, our adversaries will think twice about, you know, about conducting some kind of invasion or military operation if we know everything that they're doing. [00:31:46] Speaker 1: AI's role on the battlefield raises difficult questions, even for a company willing to debate the ethical dilemmas of its own technology. It's a conversation Amadeh seems eager to have in public. How do you handle the pressure? [00:32:00] Speaker 2: I try very hard to communicate and always be straightforward and honest. I get up in front of the company every two weeks and just talk for an hour about just what's on my mind, what's going on in the industry, what's going on in the outside world. It's totally uncensored and not only does it build trust, it's very freeing for me. Where I feel that I have 3,000 people who are on the same page as me, that is an incredible amplifier and is one of the most useful things in handling the pressure. then when we have to confront an external challenge, stand up to, you know, some very difficult situations that we've seen in the last few months, that is what allows us to have a consistent and coherent position. And I think it's an incredible advantage because I never feel like I'm alone. [00:32:53] Speaker 1: Lurking in the background at Anthropix headquarters was the surprise development of a new AI model called Mythos, a model so powerful it spooked everyone. [00:33:04] Speaker 4: Anthropix making headlines almost on a weekly basis and most notably now around Mythos. The company believes it's this enormous threat. [00:33:11] Speaker 5: Imagine a world where everyone had a nuclear bazooka, basically. [00:33:15] Speaker 1: Mythos identified thousands of cybersecurity vulnerabilities, exposing potential flaws in every major operating system. Anthropix signaled that if fully released, Mythos could hack banks, pry open state secrets, and cripple critical infrastructure. [00:33:30] Speaker 2: I think the thing that surprised me most about it was the models had been climbing in their ability to find vulnerabilities. It was a particularly large jump. Some of the early companies that we gave this to said things like, this is a super weapon. You should have to own a gun license to use it. Please don't release this. [00:33:51] Speaker 1: In an initiative called Project Glasswing, Anthropix gave select organizations access to Mythos. Even federal agencies like the National Security Administration clamored to use it, despite Anthropix's blacklisting by the Pentagon. [00:34:05] Speaker 5: I think the future is this kind of cat and mouse game where we need to make sure that the good guys have the tools that they need to defend. And then at some point, the bad guys will have it too. And at that point, we have to make sure the good guys have an even better model so they can be ready for this. [00:34:23] Speaker 1: Is it possible to stay ahead of the bad guys? Really, though? [00:34:26] Speaker 5: That's what we hope. [00:34:28] Speaker 1: The criticism is you're effectively deciding who gets access and who doesn't. Why should anyone be comfortable with that kind of concentration of power? [00:34:36] Speaker 3: It wasn't like, oh, it's so powerful and let's decide who gets the power. It was a very specific concern around cybersecurity. And so the way that we decided who to give the model to was grounded in that specific fear. There's obviously nuance to decide like where do you draw that circle? I think that's really complicated. I think we've tried to be as publicly open as possible to say we're trying our best to make this decision well, but like we might not do it perfectly. What about the folks who say this was [00:35:04] Speaker 1: just good marketing? [00:35:06] Speaker 2: You know, we have suffered enormously commercially from not releasing this model. This model has incredibly accelerated research within Anthropic and production and next models. It would do the same in the outside world if we were to release it. This has hurt us enormously commercially. [00:35:24] Speaker 1: Have you had to make trade-offs already that you're not entirely comfortable with? [00:35:27] Speaker 2: Throughout the entire history of Anthropic has been trade-offs, right? In some ideal world you would prefer to before you release the first chatbot, you know, you could spend years studying, you know, every possible thing that could go wrong with it. Now we did delay. We did delay the initial release of Claude, but, you know, we did it for a few months so everything is a trade-off. Now that we're in, you know, what I would describe as a commercially leading position, we can afford to move the dial even further towards being careful, right? That's what the mythos release was about, right? It's very hard to do something like that if you're not the leading player. [00:36:04] Speaker 1: There's this argument, why wouldn't the government take you over? Why would they let a private company control technology that's so powerful? [00:36:12] Speaker 2: So I think, I actually think that's a very, that's a very serious question and I share those concerns. I don't think the government should outright take us over. Every previous powerful technology we've seen in history was either built by the government or originated with the government. So nuclear weapons, obviously, you know, initially built by the government. The internet, GPS, cell phones. AI is the first technology that's been built in the private sector. and where government has not really had a serious role and is coming in late to the game. I think that's actually a dangerous and unstable situation. It is not the situation I would have chosen. This technology, I'm scared of companies having it, but I'm also scared of government having it. And then, you know, we need basic regulation of the technology. You know, more and more, as I've seen what we've seen with Mythos, you know, I think we need to start doing pre-release testing, required pre-release testing, testing and auditing of the models. [00:37:11] Speaker 1: This was an approach the White House rejected initially. On his first day back in office, President Trump, with the help of former AI and crypto czar David Sachs, dismantled President Biden's AI executive order seeking guardrails, instead favoring a hands-off, let Silicon Valley do its thing approach. [00:37:29] Speaker 12: We believe that excessive regulation of the AI sector could kill a transformative industry just as it's taking off. [00:37:38] Speaker 1: But with Mythos and its national security implications proving hard to ignore, the White House now seems to want to gatekeep the world's most powerful AI. [00:37:47] Speaker 2: It's very funny to me how there's a particular group of people in the tech world in Silicon Valley. They started with a position of like, you know, even having transparency around this technology, even export control. You know, this is all, you know, just totally, it'll apocalyptically destroy our potential to create the technology. it'll kill innovation. And then as soon as they see the first real danger, which I've been expecting all along, there's all this talk of like nationalization and the government should just seize it. Come on, folks, here, you're yo-yoing from like the most extreme anti-regulatory if you look at us the wrong way or destroying the industry to, you know, this completely communist, the government should grab it all. We need a more, we need a more sensible, moderate approach. That's the one we've been favoring all along because we've, we've understood the power of this technology. We're not panicking. We're not denying it. We see the smooth exponential and we're responding to it appropriately. [00:38:44] Speaker 1: The reaction to AI right now, it's intense. Anthropic has built this really loyal following. There are some people who just love what they stand for, but there have also been protests right outside their office. There's a lot of anxiety, a lot of confusion. There's some real anger right now about what's happening and it actually feels like it's escalating. [00:39:15] Speaker 3: Artificial intelligence is the next industrial revolution. [00:39:23] Speaker 1: If you look at the data, people are more concerned than excited about what's going on. They think the risks outweigh the benefits and the truth is if you talk to the people who are building this, even they will tell you they don't fully know how it's all going to play out. How do you think about the weight of this moment? [00:39:47] Speaker 2: I worry that something will go wrong. You know, are we doing literally everything we can? We're certainly trying our best. We're certainly trying very hard. What I want is to create a situation where if this set of people can't do it, it couldn't be done. You can't guarantee success, but maybe you can guarantee that. [00:40:03] Speaker 1: It's getting personal. Sam Altman's home down the street has been attacked. How is that affecting you? [00:40:10] Speaker 3: It was really scary to read that. I mean, it was, we were obviously extremely relieved that he and his family were okay. In general, I think this is a time, you know, technologically, politically, where unfortunately there's just a lot more rhetoric and words that I think can lead to bad outcomes and bad things happening. I hope that this is a topic we can all just debate, you know, as peacefully as possible. [00:40:34] Speaker 2: It was scary to me too. I mean, you know, this is like a less savory aspect of the exponential, right? That as AI gets more and more of a big thing, like, you know, it becomes just such a big deal to society. There's more attention on it. [00:40:48] Speaker 1: There's been massive backlash against social media. Countries are starting to ban it. Could that happen to AI? [00:40:55] Speaker 3: I think it's absolutely possible. If the social media companies could go back in time and see the world that they see today, would they do anything differently? I like to think the answer to that is yes. I don't know. If we sort of project some of the challenges that the social media companies have faced around child welfare, mental health, election integrity, all of these topics, we're really lucky that we're second. We view it as our job to try and proactively think about all of the things that could go wrong. Because if we don't, who's going to? [00:41:25] Speaker 2: You know, I don't know that they actually set out to do the right thing or make the world a better place. And so I don't think if they were going back, they would even knowing what they do and they certainly should do things differently. I don't know if they actually will. But we can't. This is why we're trying to get this right the first time instead of waiting for things to go wrong than scrambling to justify why it's all okay. The main way I could see AI being, you know, banned or blocked is if something really went wrong. And if something really went wrong, then maybe it deserves to be. [00:41:56] Speaker 13: You have technologists saying it's going to be amazing and others saying it could be awful. Where are you on that spectrum? [00:42:04] Speaker 5: I hope the best but plan for the worst. For me, this is the most important work I've ever done. For some people, it feels like this is the last job because this is sort of the, it could mean the end of work. And I think for other people, it's, you know, like maybe it's not the last job, but it's existential to get this right. And so there's just this kind of burden. [00:42:25] Speaker 1: If the impacts could be as significant as we're talking about, as significant as Anthropic has warned about, what responsibility do you think Anthropic has to cushion the blow? What do you owe the people whose lives you've upended? [00:42:42] Speaker 3: I think our view has always been we ultimately are responsible as an industry, the industry that's developing this technology for thinking through what are the risks? What are the bad things that could happen? And if some of those things come to pass, what is our role in helping to fix them? That is our job, right? We should not just say, well, we were just trying to grow the product and suddenly there's an entire generation of young women who have eating disorders or who have mental health problems because, whoops, we were just trying to grow the product. That's not the stance that I think any technology company should take. I don't think that's the stance that we're trying to take. [00:43:19] Speaker 1: For a company whose identity is so wrapped up in [00:43:22] Speaker 5: We want to do this right. We exist as an AI safety company. [00:43:25] Speaker 1: How can we just help all of this go well? It can be hard to understand why Anthropic is pushing so hard to advance AI while being so upfront about the dangers. In his essays, Dario lays out what the endgame looks like if everything goes right with AI, a utopian future where machines and humans work side by side. AI, an inevitable force, steered toward prosperity rather than catastrophe. To mitigate the devastation of job loss, he proposes solutions like universal basic income and progressive taxation of AI companies. But as the Amides confront the messy realities of power, politics, and profit, the real test is whether that founding mission can survive the scale of what they're building. Google started with the motto Don't Be Evil, a founding promise as the company quietly retired as it grew. You are building something incredibly powerful and stand to gain enormously from it. Why should we trust you? [00:44:26] Speaker 2: I think starting from a position of distrust, you know, if you don't know anything about me, if you don't know anything about Anthropic, is pretty rational. I think Silicon Valley has lost a lot of the world's trust and kind of has to re-earn it. And the message, you know, we're trying to send is we're actually different and that has to be earned in things that we actually do. [00:44:46] Speaker 1: I understand one of your favorite books is the making of the atomic bomb. [00:44:49] Speaker 2: That is correct. [00:44:50] Speaker 1: Do you see parallels between yourself and Oppenheimer? [00:44:52] Speaker 2: The figure I most identified with was Leo Zillard who was the one who first had the idea that there could be a chain reaction. Look, my view is we're not going to get through this with like larger than life personalities or like figures who try and be at the center of everything. In some ways, actually see Oppenheimer as a failure case as what should not happen. There's a lot of powerful actors who have interests here and the only way it's going to end well for everyone is if there is some, there's basically checks and balances everywhere. [00:45:24] Speaker 1: You've said there's roughly a 10 to 25% chance of civilizational collapse. That is not insignificant. Is there a scenario where it's something that Anthropik built that caused that? [00:45:38] Speaker 2: I mean, I certainly hope not. My view is that that probability comes from the very straightforward recipe of the technology, the existence of many countries in the world, the existence of many companies within an economy and new ones created if the void isn't filled. Like, that's a dilemma that we're in. Half of what we do within the company is try and, you know, reduce the risk as much as we can, but it's never going to be zero. Suppose there are a bunch of, you know, airline companies out there and you're like, well, I'm going to make an airline company that's safer. It can both be the case that, you know, your airline company is 10 times safer than all the other airline companies, but if someone comes and asks you, like, can you guarantee that your airplane will never crash, I mean, how could you? How could you possibly? [00:46:22] Speaker 1: But if there was a 25% chance of an airplane crashing, you wouldn't get on that plane. [00:46:26] Speaker 2: That's right. 25% is too high. We're trying to make that probability much, much lower. That's, that is, that is the goal. [00:46:59] Speaker 1: How do you find your Zen? How do you relax? [00:47:04] Speaker 2: You know, honestly, a lot of it's just exposure to it. You know, sometimes I'll just like, you know, I'll just take a weekend and I'll like play some video games, sometimes with Daniela. Me and my wife go to Italy sometimes. We have a horse there, so I'll just, I'll just sit there next. I'll just, I'll just look at our horse and I'll be like, you know, Calypso, who's our horse? Like, you know, she doesn't know about any of this. Like, she's just a happy horse. Like.