FULL INTERVIEW: OpenAI CEO Sam Altman Speaks on AI Scaling and Infrastructure Need — DRM News — AI1F

[00:00:00] Speaker 1: hello again uh if i keep doing this i understand that the uh vacancy is 60 minutes maybe i can if this investing thing doesn't work out maybe i can do that another full disclosure sam is a friend i'm also on the board of open ai so um i promise you i will not only ask him softball questions i will channel my inner bread bear and uh and uh and see gooper so sam as i said ceo of open ai one of the founders um he previously served as the president of y combinator where he helped scale some of the world's most influential startup accelerators and advise companies that reshaped entire industries at open ai sam has obviously been at the forefront of advancing artificial intelligence to the benefit of humanity helping to bring cutting edge ai tools to the hands of millions of people how many people eight nine hundred million people and businesses around the world so sam let me start with what i think is the question that is on [00:01:09] Sam: everybody's mind okay which is where are we today in the world of ai i think at some point in the last few months we really have crossed a threshold into major economic utility of these models it may have happened a little bit earlier but there was such an overhang before we figured out how to use these and we had to not only continue to make the models get smarter but figure out sort of the plumbing to make them easy to use where we're now in a world where the models are astounding people with the work they can do uh and i think this has been most noticeable in coding yep but it's also happening in science it's happening in many fields of knowledge work uh sort of with disorienting speed where people are saying like man these things that i thought were still years away are happening now and i have my job shifted from doing uh you know direct technical work or uh you know legal work to managing a team of agents doing this work this is going to go much further uh i think we are at a very steep part of the curve and right now maybe you can trust say a ai software engineer to do a multi-hour task very soon it'll be a multi-day task and then a multi-week task and not long after that i think the paradigm will shift again and it'll feel like these ai systems are just connected to your life to your company whatever proactively thinking working all the time uh and having full context on whatever they need to know and just sort of doing stuff like you would trust a senior employee to do and and do you think [00:02:56] Speaker 1: that um companies have real have a real understanding of how these systems can help them and re-imagine [00:03:04] Sam: how they do their businesses some do some don't uh certainly the new generation of startups thinks differently than any generation of startups before uh it used to be that when we would talk to startups they would talk about how many employees they needed uh now they generally don't want to hire a lot they think that'll slow them down and they're all focused on how much compute they can get you know can i reserve this much capacity can i do a cloud deal for that uh can i get this many tokens and i think that is that that that is a mental shift that bigger companies are going through more slowly but some are starting to do that uh one place i think you can see this happening is engineering orgs and product orgs talking about they're doubling tripling what they're planning to ship this year [00:03:54] Speaker 1: and that has not happened before yeah and uh you'll be quite vocal in saying that um artificial general intelligence will come sooner rather than later you want to share your views on how close we are and how soon it'll come [00:04:12] Sam: at at this point i think the definition of agi really matters some people would say we already got there some people say it's very close some people say we're kind of you know it's maybe still a year away but but in any case that word has ceased to have much meaning uh there are maybe two thresholds that we could talk about that are interesting okay number one when is there going to be more of the world's cognitive capacity inside of data centers than outside of them and that to me feels like maybe it could happen huge error bars i could be totally wrong but maybe that could happen by like late 2028 and that's that's an extraordinary shift in the world um the other one is when can a ceo of a major company a president of a major country a nobel prize-winning scientist when can they not do their job without making heavy use of ai this doesn't mean that there will be an ai ceo or an ai president uh but it does mean that that the role of let's say a human ceo when i think about my job it's really quite different you still do need a person to stand behind decisions and kind of exercise human judgment and all of the understanding that we expect out of someone running an important organization to do but the actual parts of my role that i will increasingly have to rely on an ai to do because no human can you know no human can talk to no human ceo can talk to every employee at a company every customer be in every meeting be an expert in every field and so more and more i think of these jobs will be supervising a bunch of ai providing oversight deciding how to like trust the the outputs how to provide guidance and that threshold of when you really wouldn't want to be doing your job running a large organization without heavily heavy reliance on ai i think that's another sort of interesting threshold that may take a little bit longer but probably not a lot longer and and as [00:06:27] Speaker 1: you do your job how much are you finding yourself relying on some of the agents and some of the artificial intelligence that we're developing at open ai uh it's ramping incredibly [00:06:42] Sam: quickly um if i have like a new idea for a business model a strategy shift a uh a product offering we should do the very first thing i do before i even bounce it off somebody else is to ask our tools and as they get more context i think this really is like the next big thing to happen as they can get close to full context of our company access to all of our internal docs communication code customer data everything the quality of the answers thought whatever you want to call it gets [00:07:16] Speaker 1: better and better right okay so let's let's shift um a little bit two weeks ago um you announced a 110 billion dollar funding round i asked chat gpt how does that control compare to any other fundraising that's been done in the public markets i actually don't know uh four times as large okay the largest public offering ever done was roughly 25 billion that aramco did several years ago and the public market that's supposed to be the broadest and deepest sources of capital okay three strategic partners amazon nvidia and softbank tell us a little bit about this how is this an inflection point of the company and one of the questions somebody asked is what are we spending all that money on the there's many hard parts of this [00:08:10] Sam: business but one of the hardest ones is the infrastructure is so expensive you need so much of it and you have to commit so far in advance i have never seen any other industry quite like this i mean there have been clearly many capital intensive industries throughout history but as i look at the what's to come in front of us in the few in future years if the ramp stays as steep as it looks like it is right now the demand is growing as fast as it's growing you have to do some pretty unusual things uh open ai does a lot of things that look weird we spend a lot of money on infrastructure in advance of revenue we do new business models like ads that seem like you know maybe not the most profitable thing we could do um a long list of other things but we have this fundamental belief in abundance of intelligence and that one of the most important things in the future is that we make intelligence you know to borrow an old phrase from the energy industry that didn't quite work too cheap to meter we want to flood the world with intelligence we want people to just use it for everything we want this to just be something that the future generation doesn't think about they expect everywhere and everybody has access to like geniuses as many as they need in any area that they need and this principle which is one of our kind of like top guiding principles does lead to a lot of behavior that would look less natural for other companies and one of those is we really want to get out of this world that we have been in that we still think we're on a trajectory to stay on without changing what we do of always being capacity constrained right and capacity constraint you mean compute yeah and i've [00:10:02] Speaker 1: often had you say a lot that compute is revenues you want to talk a little bit about how you think how [00:10:09] Sam: you think about that fundamentally our business and i think the business of every other model provider is going to look like selling tokens you know they may come from bigger or smaller models which makes them more or less expensive they may use more or less reasoning which also makes them more or less expensive they may be running all the time in the background trying to help you out uh they may run only when you need them if you want to pay less they may work super hard you know spend tens of millions hundreds of millions of someday billions of dollars on a single problem right that's really valuable but we see a future where intelligence is a utility like electricity or water and people buy it from us um on a meter and use it for whatever they want to use it for the demand that we see for that seems like it's going to continue to just go like this and if we don't have enough we either can't sell it or the price gets really high and it you know kind of goes to rich people or society makes a bunch of sort of central planning decisions that i think almost always go badly about you know we're going to use our limited compute supply for this and not that so the best thing to me throughout all the history of capitalism innovation whatever you want [00:11:31] Speaker 1: is to just flood the market yeah and obviously a critical part of addressing that uh compute demand is stargate and this is an infrastructure conference and you know you announced that several months ago or almost a year ago i guess it was yeah how's it going in the us and then how is it going because i also started in uh in abu dhabi yeah yeah um honestly there's many cool parts of the [00:12:02] Sam: job one of the coolest is getting to go visit these mega data centers under construction and operation yeah just the scale is you know of like these gigawatt campuses it's really hard to explain you see these photos and it's like okay it does look big and then you go there and you're walking through it yeah building to building and it's just you know 10 000 people there all these different skilled trades doing all these different things and you know it looks like a spaceship inside it's really quite incredible um we are training right now uh on the first site enabling uh what i think will be the best model in the world uh hopefully by a lot and it is it's so amazing to have gone from you know many visits while it was under construction uh and just really internalizing the scale and incredible complexity to the day that one researcher at open ai types in like one command and it pushes enter an unbelievable number of gpus spin up and start doing this one huge computation [00:12:58] Speaker 1: all together yeah it's very cool and and um what have been the most pleasant surprises and what have been the hardest things about getting it up and running and scaling it going forward uh i mean there have been [00:13:14] Sam: all the expected challenges and then the unknown unknowns like there was a crazy weather event in abilene uh that we hadn't you know it was outside we had planned for and that brought things down for a little while um there's all the supply chain challenges anything at this scale it's just like so much stuff goes wrong lots goes wrong lots goes right but the just trying to build something around the clock with such complexity there's all the stuff that goes wrong one of the biggest surprises on the upside is how many different organizations had to come together to do this in an incredibly short period of time uh and how much we all like ended up working together as one team uh under a lot of pressure [00:13:56] Speaker 1: uh and of course the power demand part of it is the one that a lot of people are focused on um are you optimistic that we will solve that challenge first in the u.s and then we can talk [00:14:11] Sam: about it you know in other places uh in the long term i am okay uh i have no doubt we will figure out how to build huge amounts of power generation ai will help of course uh but i think the portfolio the world has in front of it gas solar nuclear efficient nuclear fusion more like we're i feel good about what what we will be capable of and what we will eventually do um given the demand growth we are seeing i am sort of hoping for a miracle in terms of figuring out how we can get way more efficient per watt with models to give us time to build out all this infrastructure now the track record there has been incredible um people cite whatever amazing statistic they like about how much more efficient our models have gotten over our industries models have gotten over time but one that i think is incredible our first reasoning model was called o1 came out like 16 months ago uh and our latest model where we've now integrated reasoning is 5.4 to get the same answer to a hard problem from that first model to 5.4 has been a reduction in cost of about a thousand x maybe i was a little bit wrong on the timeline maybe it's a little bit longer but in any case since oh one until now yeah a thousand x uh that is unbelievable in a relatively short period of time and the thing that that the two things that i think that points to one we are still so early and in this paradigm and we're still so we have so much more to gain about our understanding of how to develop these models and train them and run them efficiently that we have more than and still are doing things in dumb ways and will get better and better and two is that human ingenuity and the ability to operate in constraints and to find ways to solve problems almost always surprises you on the upside so it's not just that the models have gotten better it's that we have figured out you know kernel engineers came to help figure out how to write more efficient kernels and like power engineers and the people that design data centers found more efficient ways to do that so people are answering the call well beyond just the model side right make this more efficient right and then of course um [00:16:48] Speaker 1: obviously open ai is a big customer of the nvidia's and amd's of the world but and you know i think we signed a big contract with amazon but we're building our own chip yeah what's what's the thinking [00:17:00] Sam: behind that uh so the chip we're doing is inference only okay and the thinking behind it is that on this you know come rise up to solve problems in front of us um we think that a specialized chip to be not necessarily the fastest inference chip but the cheapest inference chip the most efficient per watt given the constraints we see in front of us is going to be important for all the agent demand we see in the future so it's a it's an opinionated bet it's a limited chip but the thing that it does in a world where we're energy constrained i think will be very important maybe you can explain because [00:17:35] Speaker 1: i'm not sure everybody in the audience is uh ai proficient with the difference between an [00:17:40] Sam: inference chip and the training chip sorry i should have done that first so there there are two main phases to uh ai workloads now they're going to blend together eventually and it'll all be one continuous thing but right now first we train a model you know a gigantic number of gpus crunch for weeks or months on a bunch of data and you can think of that like maybe it takes you 22 years in life to get your education and you learn a bunch of stuff starting when you're a baby and you drop things and see they fall and then eventually like in college physics understanding like at a very detailed level what's going on there um and then after that if you ask a model to solve a physics question it that's called inference uh and that's quite efficient but this is true for people as well and when people talk about all these ai models are so efficient they're usually comparing the 22 years that a human takes to train to the one second that an adult takes to solve a physics problem if you compare the you know the model solving the physics problem to the human solving physics problem actually the models already probably are more energy efficient but training is this hugely massive amount of processing power that then produces really just like a file of numbers that you can then pose a question to and get a response and are you optimistic about the progress we're making on the chip yeah um we should have the first chips deployed at scale uh by the end of this year we should have the first chips back just in a few months now um and it looks like it'll be really good fantastic so you announced a new partnership [00:19:21] Speaker 1: this morning actually with the north american building trades unions to expand training pathways for kill skill construction which has been a subject of many of the discussions today can you tell us a little bit about what you and sean mcgarvey yes agreed and what's and he's going to be on stage later on so let's hear your version of it and then see if his version is the same so [00:19:48] Sam: we talked earlier and we've talked the world has talked many times in the past about the need for ai infrastructure the need for the world to have the physical infrastructure power plants transmission lines data center halls chillers obviously the racks and the gpus and everything that goes inside of those um and i i sort of wish everyone would go visit at some point one of these mega scale data centers because it it's hard you know when you ask chat gpt a question you get your answer back and it's really hard to visualize the scale of what it took to make that or what it takes to make that happen uh people talk about all of these different places that were limited we are limited in the you know number of turbines or now it's the voltage transformers or it's the memory fabs or it's building the data centers whatever all of these things have one thing in common which is they are massively complex physical infrastructure that require skilled trades people uh and a lot of them to do and no matter where the choke point in the supply chain is at any one time when i talk to people about what it would take to accelerate it um it is more skilled trades workers to build out all of this infrastructure that we all depend on i think these will be incredible jobs uh more than that i will i think they will lay the foundation for the next generation of american infrastructure and economic prosperity and we are thrilled to get to [00:21:24] Speaker 1: work uh together to to drive that faster just so you know we at blackrock share that view and we think it's really exciting that that you announced this partnership today we think that's really really terrific i think another question that's on people's minds is competition with china where do you think that stands and what what do we need to do to make sure i assume we're ahead but i don't know so you might tell us whether we are or not what do we need to do to make sure we stay ahead [00:21:57] Sam: so a general framework thought first and then i'll answer the question more specifically i think the discovery of deep learning is is closer to discovering an element or a fundamental property of physics than it is uh of like a secret technology um and that means that eventually and eventually probably not being very long the fundamental ideas that make a model so capable will be simplified they'll be very well known and just like we kind of understand how big parts of physics work we will understand as a scientific principle how big parts of artificial intelligence work um this is this really we started to appreciate this with the scaling laws uh that open ai published maybe seven years ago now um there was such a measurable beautiful correlation between the resources that go into a model and the intelligence of that model that it kind of felt like hair raising but clear at the time that there was just something fundamental going on here as a scientific principle now there have been a lot of details we've discovered since there will be more to come but like other scientific frontiers it is simplifying and becoming more clear over time and eventually this recipe will be well understood as a scientific principle it will not be a trade secret in the sense that other things have been now the analogy i like best historically uh from technological history is the transistor transistor was also a sort of fundamental scientific breakthrough very hard to discover kind of chancy to discover uh took us a little while to refine our discovery but once we understood it the scientific principle was clear to everyone there were still massive amounts of operational knowledge that went around that uh tsmc can still do things that no one else in the world can do and you know i i expect the industrial process around this to have a lot of advantages uh competitive advantages i also expect that the integration in workflows and training data and other and usability of models there will be a lot of differentiation there maybe most of all i expect there to be differentiation on who has the infrastructure and how much of it but the fundamental scientific principles i are going to be well known and they will fit on a t-shirt i think um in terms of where we are uh the most capable models in the world the frontier the u.s is leading on um the cheapest inference usage for a two-generation earlier model china is leading on um infrastructure the u.s is currently leading on but china is moving much faster on and that kind of industrialization productization whatever you want to call it i the u.s is leading in closed source china's leading in open source uh i think the u.s is probably leading overall and you were recently in india and um sound like you're [00:25:11] Speaker 1: very excited about how india is thinking about the challenges and the opportunities i was blown away [00:25:17] Sam: uh talking to indian startups and how they are using this technology um i like got there and someone like handed me a briefing sheet for india and it was like you know codex usage in india has like 10x in some small number of months and i was like that's got to be a bug like that can't be right it can't be right um but it was true and the and then i started talking to these startups and it's an even stronger version of what's happening in the u.s of people saying hey the world is different um you know you talk about a one-person startup i'm trying to build a zero-person startup i'm trying to just write a prompt that's going to make my whole startup and you know write my software do my customer support do my legal stuff whatever and then i want to go on vacation um and the companies big companies in india that were just saying like how much capacity can we buy from you how long can we reserve it for can we negotiate this right now we're not going to like let you leave the room until you agree on this with us that just the level of aggression and speed and sort of a belief that ai was gonna reshape the business landscape in india was was really quite impressive um yeah is that different from when we talk to customers in the u.s uh it's it's it's the same vector but they seem a little further along okay or moving faster yeah [00:26:50] Speaker 1: the other the other comment i think i saw you made was that there's a difference between autocratic and democratic ai what did you mean by that and what do you think is at stake [00:27:05] Sam: once in a while i think a technological shift comes along that reshapes society to such a degree that it does not decisions about it do not belong to the handful of companies that happen to be developing it i am a huge believer in capitalism i am a huge believer in the rights of companies shouldn't interfere too much but i think this is one of those exceptional times where society has a legitimate interest in what the impact of this technology is going to be i think the internet was one of these times too and i don't think we got all of that right but i'd like us to learn what we can do better and if what the ai companies say and i also believe it comes true and that this is going to reshape the economy this is going to reshape geopolitical power this is going to change how we all live our lives then i don't think it should be up to companies or a government to impose a particular will on how this is going to get used i think that this belongs to the will of the people working through the democratic process and companies like ours i think are quickly more quickly than companies of previous generations have had to do moving into a sort of critical infrastructure role right where we have to say we create this technology we are experts in it and we should have a real voice and we have a we have opinions and understanding of where its limitations are and where it's not ready to be used and where great harm could come uh but the rules the limitations have to be agreed upon by society through this process and because the technology is moving so fast it'd be great for that democratic process to run a little bit faster but companies governments uh need to be able to depend on companies like ours to integrate and be able to use the the technology okay so come back to the global ai race [00:29:33] Speaker 1: where where where do you think the u.s is the most vulnerable [00:29:44] Sam: three three things come to mind uh one there's been a ton of noise made about the global supply chain independence and us infrastructure i don't have anything new or deep to say there but i can't overstate how scary this is to me yeah um the if we fall behind on infrastructure and can't catch back up um if if if globalization falls apart in any of the many ways it could uh and we are not able to fairly independently keep building ai infrastructure that that seems like a big vulnerability and i don't i don't hate but i don't love our global position right now um the second is if we don't it's a competitive world and if we don't move as quickly as other countries on economic adoption of this uh then i think we will lose the advantage that we have from being the economic powerhouse that we are and this is about how quickly companies adopt it this is about how quickly our scientists adopted this how quickly our government adopts it uh this is like on the positive side i think this is a once in many generation opportunity to really improve the economy yep really rewrite some of the rules of society that aren't working in light of this new incredible wealth fountain we have um so i can see the world where this is not a disadvantage at all but this becomes our biggest competitive advantage i don't i don't think it's like super obvious that we're on the trajectory we want to be there now again i don't hate it i just i think it could move faster and and you can see a bunch of potential headwinds you know like ai is not very popular in the us right now data centers are getting blamed for electricity price hikes almost every company that does layoffs is getting is blaming ai whether or not it really is about ai yeah um there is this real debate about the relative power between governments and companies going on so so there's like a lot of there's a lot of stuff happening there and then the third category is uh diffusion into the rest of the world is the world mostly going to build on the american ai tech stack chips models applications whatever or are we going to enact a set of policies that make that harder it sounds to me like you think [00:32:20] Speaker 1: that ai could be the foundation of an immense productivity boom if properly used and if adopted [00:32:31] Sam: yes for sure although i think the way we measure that is going to have to change um explain i can see a world where we have an incredible productivity boom quality of life goes up and up uh most of the things that we think we we say we would care about get better and better and yet gdp and the way we currently measure it goes down and down like deflation for a very long period of time right and i don't know what it means to like live in a forever deflationary world i don't know what it means to think about gdp in a and gdp's correlation of quality to quality of life in a world where you know more of the economic capability or the sorry intellectual capability is inside of data centers and outside of them but you know maybe we're going to find out and and i think there's going to be a lot of debate in the coming years about what the right [00:33:37] Speaker 1: thing to measure is do you think we're thinking about these challenges and issues the right way are [00:33:44] Sam: we having that are we starting to talk about that i i think yes we are starting to um if there was an easy consensus answer we have done it by now so i don't think anyone knows what to do um the all of these things that we've depended on for so long as a society is sort of coming into question at the same time uh there was this quote that i saw on the internet a few weeks ago it's really stuck in my head it was something like for for centuries maybe millennia we have learned a lot about how to structure society to manage scarcity almost none of that helps us as we have to quickly learn towards managing abundance uh so that's like a real change to how capitalism has worked um capitalism has also depended on this balance between somewhat of a power balance between labor and capital but if it's hard in many of our current jobs to outwork a gpu then that changes and i see we're out of time but i had like a list of 10 things like that changing yeah all at once i i i'm not a long-term jobs doomer i i think we will figure out new things to do i'm also certainly not a long-term capitalism doomer i believe in it very very deeply um but i think the next few years are going to be a painful adjustment as we get to this future we all get to redefine of what the new system and just this incredible prosperity looks like and there are going to be some very intense and uncomfortable [00:35:19] Speaker 1: debates on the way there okay so thank you sam for that very thoughtful discussion here's what i will say five years from now we will come back here and we'll see where we are and how we have navigated our way through this deal deal i look forward to that country thank you thank you thank you [00:35:47] Sam: we'll now take a short break please join us in the east green room