Who are all those AI data centers for?

[00:00:00] Speaker 1: good morning artificial intelligence is a critical industry and our thesis is that almost all the applications for AI will happen here in Asia and that most of the massive investments in data centers in North America and in Europe are misdirected and most of those investors will lose money in the United States data center construction and the related CapEx makes for the biggest economic bet in history spending for AI in the past eight years on a percentage of GDP is bigger than railroads more than three times larger than the construction of the interstate highway system all of that previous spending did give the United States canals and railroads and electricity highways and the internet and many of those investors did lose everything they went bankrupt because they were early they built out capacity for customers who would come later AI is probably very different though railroads need to be where people are same with highways and cities sprang up alongside them that is not true of AI some of the most critical artificial intelligence operations can be done anywhere in the world by users anywhere else in the world and that's the problem the entire economy is dependent on the success of those data centers paying off high levels of debt are required to build them along with capital raises from the sales of equity they also present an opportunity cost data center construction is crowding out home building for example we're desperately short of affordable housing but hyperscalers are snapping up construction workers to build data centers instead of apartment buildings land prices are going up and that's bad news for companies who want to expand industrial production this is a map of where the data centers are in operation right now bigger dots are bigger centers in terms of compute and power demand but this map is for planned projects and the dots are a lot bigger pulling a lot more power at a time the grid is already under strain and the industry is selling that promise right there silicon valley and wall street are telling policy makers that AI will bring in a new industrial revolution ten times bigger and faster scientists point out how all this new computing power will lead to new drugs and advances in robots and other technologies but already that doesn't make a lot of sense the robotics industry is here in Asia it's china and south korea and japan who build the robots the supply chains for robots run through china china and india account for most of the world's production of apis the active pharmaceutical ingredients that make the drugs work we're saying here that the advances in robotics will benefit the people who build robots and the factories that use the robots new drug development will happen in the places where labs can build new compounds and test them on patients and when we see the statistics for example on the relative computing power of the united states compared to china we're measuring the wrong thing chinese companies are catching up on the chips and have already passed western firms on most of the rest of the ai stack but what's actually more important is that it's here in china and elsewhere in asia where the actual use cases for ai are right now and will be for the rest of the century that's a good question if it's true that three-fourths of the world's compute is coming right now out of the united states and china is about a fifth of that where does that leave everyone else this is one of our favorite maps and it helps us make sense of the rest of the world yushi is a city 50 miles behind me kunming airport is 20 minutes in that direction where i'm standing right now is four hours by plane outbound kmg to over half the population of the world kunming to kl is due south four hours away and malaysia is home to one of the largest ai data center build outs on earth powered by the world's most powerful chips and plugged into cables between china and the rest of the world malaysia is a brics partner country and so is very friendly with china and chinese firms are making big investments there malaysia also has a good relationship with the united states and so can buy chips from nvidia and attract investment by the hyperscalers equipment that serves us and chinese companies sit next to each other there are obvious cost advantages to build and operate a data center in south east asia the mid-range cost of construction per megawatt in malaysia is 9.6 million dollars the median construction cost for an ai data center in the united states is more than twice that that puts this headline into better perspective malaysia approved 27 data centers as of july and only one of them is online right now so 26 ai data centers are still coming worth over 50 billion dollars but that translates to around 150 billion dollars in comparable compute where they built the united states said another way ai data centers in malaysia cost far less to build and investors can get their money back far faster those lower costs are going to immediately take market share from data centers in the united states in europe there are many tasks in ai that need to be performed by data centers who are as close to the end user as possible any operations that require low latency all that business will go to the data centers that offer the fastest response time this report lays out the latency problem for anything like real-time language translation or smart minds or smart ports or smart cities with self-driving cars it takes too long for signals to go across the ocean and back the key metric for that is time to first token and it is the metric that users experience themselves on the same continent or in the same country ttft is 5 to 30 milliseconds but a query or task running from the u.s to singapore or the other way around takes much longer it's 6 to 30 times slower so those data centers in north america will keep all of that business for the applications that demand low latency in north america but the data centers in china and across southeast asia will retain all that business here too for the same reason they need to be close to where the end users are and in asia there are just a lot more users that 200 millisecond fiber optic round trip will cause a self-driving system to crash a smart grid might fail because the vast majority of the world's heavy manufacturing and logistics and population sits inside that circle all the latency sensitive ai infrastructure must be built directly alongside it western data centers are geographically locked out so a developer in vietnam for example is far better off using china's token outbound using huawei chips in a mainland china data center compared to the newest and fastest ships from nvidia if it's way over in ohio or texas it's faster but there is another major issue which is that lots of functions for artificial intelligence are not time sensitive for example batch inference is collecting thousands of requests and waiting for the lowest cost times to process them online catalogs and product and pricing updates video and media generation and processing back testing trading models weather prediction software indexing software code written years ago but still running systems today the data crunching of massive volumes of medical journals and engineering reports and science papers and patents into usable databases that can be then turned into real products and medicines and industrial processes all of that business will migrate eventually to the lowest cost compute into the data centers can do all that data crunching at lower prices these are issues that wall street investors are missing the fastest chips don't really matter nearly as much as the fastest turnaround time and the developers for many of the very high value added functions in ai they don't need the fastest chips anyway they want the lowest costs and the price advantage is what malaysian data centers will likely always enjoy over ai facilities anywhere else so it's hard to see who exactly these data centers in the united states are being built for and how investors building them can possibly get their money back this is xiondu scenic area in zhijang be good [00:10:28] Speaker 2: Thank you. For more information, visit www.FEMA.gov.au