Inside one of the world’s most efficient Data Centers

[00:00:00] Speaker 1: If you're like me you do a lot of Instagram stalking but how does the Instagram profile actually load on your phone? How does a video on your Instagram feed actually get sent to the device? I'm here at one of Meta's data centers in the United States to learn all about how the technology inside of these facilities actually works. Now you might be wondering why does Meta build their own data centers? Well when you're operating at the scale that Meta is operating at you need the efficiencies that are that come with creating your own custom designs. So what that means is that inside of this data center facility here there are bespoke designs around how the equipment works to power the different services that Meta operates. I'm talking Instagram Reels, Instagram Stories, Meta AI, Facebook News Feed. Honestly I could go on but I want to get to the point of the video. So what are the different types of designs that Meta is using inside of its data centers? Well it entirely depends on how detailed you want to go but at the surface there are four types of server inside of this facility. Let me break it down for you. First compute. This right here is one of the core building blocks of Instagram, of Facebook and of any of the Meta services. This server here is used to process compute. That's requests that are sent from your device to the data center when you refresh the page or you upload a new photo. Now this might not look that impressive to you watching this. However there are around 80 of these in a single rack and hundreds if not thousands of these racks in this single building and obviously across the campus the many buildings that are here. I find it really interesting how the big gray box that you see on a landscape which is called a data center. Actually just has lots of these individual processing units, processing individual requests one at a time. Put that back so we don't break it. I'm gonna leave it there and call over the engineer. It's compute servers like these that will be responsible for handling logins, refreshing the feed, handling messages across Facebook, WhatsApp etc. with some variance in the designs depending on the use case but all falling within this compute category. Next, storage. This single rack here will have around 20, 24 petabytes worth of storage just in this single cabinet here. The entire facility measures its storage capacity in exabytes which is the measurement above petabytes so it is a massive amount of storage. To put that kind of storage capacity into perspective that is around half a billion movies and that is stored in just one facility. But you see the thing is that your Instagram video might be stored in this hard drive here or it might be stored in this hard drive here. But it's it's not only going to be stored in that singular hard drive. And the reason why is because hard drives fail over time. So you need to implement redundancy to make sure that you still have access to that file even if the hard drive that it is on fails. Now I was asking them about this earlier and they said that they would roughly keep around five copies of a single file in a room like this which is a lot. Five copies of any piece of data is brilliant redundancy. But of course what happens if something happens to this data center? They would also keep multiple copies around the globe so that if this entire data center vanished you would still have access to all of the data and you as a user would you wouldn't even notice. It's crazy to think that there is that level of protection against your data, your content. I'm very impressed. Now on to the two server types that are the most exciting if you're a fan of AI. And honestly I didn't expect to see this when we first got the invite to do this tour so I'm pretty excited about it. This is a rack that's used for training AI models and inside of it there are 16 of these Nvidia H100s. These are very expensive. I'm very terrified of holding it right now but that's beside the point. That's not the only equipment that they have inside of this rack. They also have a compute tray which handles all of the computational effort of deciding what data gets sent to these GPUs. You have the input output tray which takes the data which receives the data coming in to the rack and then sends it back out again. You have the network switch which handles 1.6 terabits of throughput. That's a lot of data. It also has one terabyte of memory. That is a huge amount of memory by any standard. And then like I said you have 16 of these Nvidia H100s which are used to train the models. Now they aren't working alone. They are all connected by this network switch so that the tasks for training can be sent to each of them really efficiently. The final server type is AI inference to keep things short because this topic could easily become an entire video in itself. AI inference is when you are sending an input into an AI model to then receive an output. This could be asking a question to an LLM for example. The hardware behind this process is vastly different because instead of being beefed up to train using vast amounts of data they are instead optimized for receiving an input interacting with the model and then sending an output as quickly as possible. So now that I've talked to you about the four different types of servers that are operational inside of this data center I want to talk to you about the infrastructure that supports them because when you're building a data center and you decide I want to put these computer chips inside I want to put these amount of storage capacity in that's great but how are you going to support it with power with cooling with networking these are all complicated topics. First let's look at power. Now when I asked to see the power supply for this building and we walked into this room I thought that this would be it. It's not. This entire room and the many others of duplicates of this room around the site is the power supply for the building and that's no surprise considering that these facilities they do use a huge amount of energy. These systems aren't plugged into just a regular outlet like you would get at home. Meta connects directly to the grid and it pays 100% of the cost of doing so. In fact Meta actually invests hundreds of millions of dollars per annum in upgrading the infrastructure to allow them to put these facilities into place but those upgrades also benefit the local communities surrounding the data centers. These upgrades aren't simply about how power is collected it's also about how it's stored and about how it's transported across the country in the U.S. Because if you didn't know and I didn't know this the main grids in the U.S. are massive spanning hundreds of miles. So what that means is that the power source whether that be nuclear, solar, wind, whatever you want to use can be hundreds of miles away from the data center location. And that really matters because it allows infrastructure to be spread out and put those power sourcing locations in a place where you are going to get the most efficient sourcing. You put the wind farms in the windy places, you put the solar farms in the sunniest places but that doesn't mean that you need to have a data center right next door to those solar panels. Now if the power supply from the grid fails that is a problem but it would be a bigger problem if we didn't have a backup. Fortunately we do. We have backup generators. Outside here not these little things here these are called transformers. This massive structure here is a backup diesel generator and there used to be 20 of these per data center building but thanks to efficient design there are now only four. Let's have a look inside. It is warm in here I'll tell you that. This is one of the backup diesel generators and this will kick in within around 90 seconds if the main grid power supply fails. That 90 seconds though doesn't result in an interruption of service because each one of the server racks inside of the data center has a small backup battery which can help to aid in that transfer process. Now on to cooling. This entire story of this building here is dedicated to keeping servers cool and there's actually a few misconceptions around cooling and water usage within data centers. One of those misconceptions is that millions of gallons every day are being used by data centers. Another misconception is that there is wastewater that the water is somehow dirty once it has passed through a data center and that just is not true. This entire facility here not just this building all of the buildings on campus actually has zero water usage for up to eight months per year. That's zero water usage for the data center equipment. Of course water is still used for flushing toilets and cleaning and things like that but this misconception that millions of gallons of water is being used to help cool these servers just isn't true. Step one of the cooling process is getting air into the building and that's done using these massive air vents that go all the way along the side of the building. That's bringing in air from outside. Step two is making sure that that air is clean because there are things like dust particles, pigeon feathers, and whatever else that you may not always see inside of air outside but it is still there and I can prove it to you by showing you how much dust is on these filters here. I've got to make sure I don't put that on myself. Ultimately you don't want that dust getting on your computer hardware because then that is a massive cleaning job but also it's going to get much hotter so if you take all of the dust and all of the dirt out of the air here you can help to make your data center more efficient while it's operating. Step three begins exactly where we left off which is the clean air coming through these filters and then being passed into what looks kind of like a honey honeycomb structure. This honeycomb structure isn't really in use at the moment because like I said water isn't being used throughout these certain months within the year but when it is water is being pumped into this honeycomb structure to allow the air to pass through it and cool the air down. For a little bit more context as to why this method is being used this entire data center is air cooled so liquid is not actually being ran through the server racks it is purely being cooled by air so the use of liquid in this data data center is here to actually just cool the air down a little bit. For the final step of the process all of this lovely cooled air is being sucked into this room and down into the data center floor here so below me right now are hundreds if not thousands of server racks. The result of that cooling is that well it's not very warm in here in fact it's actually quite cold and that's because the cool air is coming in and it's going through these servers into what is known as a hot aisle. So after the cold air has gone and passed through the server to cool it down that hot air needs to go somewhere and it goes into here. If you look up you can see that the hot air goes up there and then out of the building. I'll be honest it's very loud inside of the hot air which brings me on to the next topic noise pollution. There's a belief that data centers are very loud and if you're standing inside of one it can definitely seem that way but on the outside they really aren't that loud at all. And here's how I'm going to prove it to you. We are outside on having a lovely little tour in the back of a truck and you can't really hear anything in fact what you can hear is birds a bit of wind and the interstate which is over there I believe but considering that they are massive air vents on the side of the on the side of the building. So now that we've talked about servers we've talked about how they are powered we've talked about how they are cooled we now need to know how the video that you're wanting to watch on Instagram gets all the way to your fun. That's right now it's time to talk about networking there are multiple steps to this process and it starts here in the networking room for this specific cluster so there are let's say a few hundred servers inside of this room for example they all need to be connected into their own mini cluster but it doesn't just stop there because there are more rooms like this across the campus so they then need to be connected using the building distribution framework. That connects all of the systems that are on site to each other for their different purposes but then of course we still need to connect that to the internet and that is done through a main point of entry not just one entry though there are multiple entries that are put in place for redundancy purposes because well if that one entry was to go down or break or something was to go wrong your entire data center would be useless and that would not be very good. So you need multiple methods of getting that data in and getting it out the eagle-eyed of you will have noticed that there is a lot of cables behind me but what's actually inside of them? It's these things these are individual fiber strands which are inside of this massive networking cable that's used for the inter-building connectivity here on campus and it's really interesting because there are 3,000 of these individual strands in one of these cables and of all of the networking cables. And of all of the networking cables on this campus that's the thick ones like this or the thin ones that you saw in the server room there is enough cable to stretch around the world multiple times over. Building these data centers takes a very long time because there's a lot of work that goes into building them and the reason that you build them is because either you need additional compute capacity you need redundancy by putting data in multiple locations or you want to lower latency by building a data center closer to where a lot of your customers might be. Now you can plan for these things but sometimes you need additional capacity very quickly. So to finish off this tour I wanted to show you something that Meta showed me that blew my mind. Tech is a really fast moving industry and what that means is that sometimes the lead times of a data center are too high. There are things that you can do to combat that and one of those things is by building a rapid deploy structure. The lead time for building one of these is months whereas the lead time for building a data center like one of the massive structures that we've seen today is years. What that means is that if you're developing a new product or you're building a new AI model because this industry moves so fast you're able to move fast with it as opposed to being like right well now we've got to provision all these resources break ground on a new data center site etc. It takes so much time but this this makes it a much quicker process. So let's have a look. Inside of this rapid deploy structure there is near enough the exact same setup that we saw in the main building. Only a slightly smaller version of the calling system and you also can't fit as many non data center related facilities inside of it. Inside of this structure I kid you not there were thousands of server racks from the very start of the room all the way to the end. They have packed this in to have maximum compute capacity per square meter and that's really good because it means that they are using the space efficiently. I'm really grateful to Meta for inviting me to have a look at their facility. I know that being allowed to film inside of a data center is not that common and being given the amount of access that we were is a privilege that I am really grateful for. So thank you to Meta and I hope that you have learned something in today's video.