Tour the Allen Data Center (HD) - Cisco

[00:00:00] Speaker 1: This place is fascinating, I mean, look at how bright and large and just built out this place is. So, our first field trip to cover a physical data center, couldn't be a better one. I mean, as you're looking around, this is Cisco's first from the ground up data center out of 50 some odd data centers. They've got 13 of which are in production. This is going to become one of the ones redundant in an active, active configuration with the Richardson data center, DC1 to DC2. [00:00:46] Speaker 2: Yeah. [00:00:47] Speaker 1: But as you're walking around here, what's jumping out at you? What kind of stuff do you think is valuable? Because we don't get to cover this kind of stuff normally. [00:00:52] Speaker 2: I love that this place is really close to some good restaurants. Okay. I think that's a big piece here, too. [00:00:56] Speaker 1: That's a good point. [00:00:57] Speaker 2: All right. But besides that, I think the thing that really sticks out to me is how warm it is in here. Restaurants. You know? [00:01:03] Speaker 1: I mean, this is-- Okay, I thought it was me. I thought it was hot flashing. [00:01:06] Speaker 2: This is warm, isn't it? This is the effect of the new ASHRAE standards that say that we can go from 65 to 78 degrees, which is quite a jump. Huge jump. Yeah, absolutely. So, you know, the ability to make that kind of jump really saves us a lot of cash, but it allows us to really plan data centers out a lot differently. So, I mean, if we could start kind of at the ground floor, you'll notice that, you know, we're solid floors here. There's no raised floor. And the advantage that that gives us is, number one, we don't have to immediately plan for what our end game is, what our data center is. What's our final capacity going to be? We can say, these are solid floors, and then we just kind of grow as we need to. [00:01:44] Speaker 1: Let's talk about the additional reasons why they would do that, because the ASHRAE standards actually give us a wider range to play in. Yeah. In this building, which we'll talk about further with air-side accommodizers and different things that they do from a green. They're pursuing a full gold leads certification, which is no small matter when it comes to-- [00:02:00] Speaker 2: There's no marketing fluff to that. Gold leads, that's pretty hardcore stuff, man. [00:02:03] Speaker 1: So, I don't think they've even got it. So, you can't even claim that yet, because there's really everything from construction and design all the way through actual production. And so, we'll see examples of that throughout. But talk about air flow, because you mentioned the hard floors. Notice there's no ceiling as we look up here as well. What is it that we're seeing from a design perspective that was, is it because ceilings are expensive, or there's probably an air flow consideration for this, right? [00:02:25] Speaker 2: Well, what we did was, is that we worked with-- Now, see, you're saying green. I'm not a big fan of green. You know, I think that green is kind of-- I think it's a very overused term. But the principle we're talking about is the same. [00:02:34] Speaker 1: Don't beat me up on the word. [00:02:35] Speaker 2: But the principle of living in harmony with the natural order of things works really well here. So, if you look up in the ceiling here, is that, yeah, there's no hard ceiling, you know, per se, no tiles or anything like that, but we're using the ceiling as a plenum. Yeah, ah, good one. You know, so then if you switch over here to the server racks, then what you see is we've got chimneys on every single one of these racks. What it's doing is, is it's really allowing that hot air, that, you know, that 120 degree air to rise naturally, go up into the ceiling, the ceiling acts as plenum, and it photos it off, and then we're using fresh air. As you can see the white ducts here next to the LED lights, the fresh air is kind of swooping down, and it's being sucked into the equipment. [00:03:17] Speaker 1: Because it's going to fall naturally in that cold aisle. Absolutely. The equipment fans are going to suck it through. The fans are going to pull it through. The chimney raises it, and it's purposely, it looks like the chimneys end purposely from a stratification perspective above where that cold air layer is, right? Absolutely, so then you don't have rain. [00:03:29] Speaker 2: Yeah, you don't have rain in your data center. [00:03:31] Speaker 1: Literally people have rain? Yeah, yeah, yeah. That seems like that would be a bad thing. [00:03:34] Speaker 2: Yeah, you definitely don't want that. That's a big problem, especially with some of the older DCs, is you have that mix and you have rain kind of coming there. So this actually kind of prevents, and you've got this nice cyclic action of that hot air and cold air. [00:03:45] Speaker 1: So while we're still talking about water in this ceiling for a moment, I also noticed what appears to be extremely standard fire suppression. I mean, seeing, to me, this would make me very nervous because you're seeing regular sprinkler heads. What's the story with that? [00:03:56] Speaker 2: So that's something that really caught me too when I was in here, is that when I walked into this data center, the first thing I started looking for was the respirators, because we're using like FM200 or HALO. [00:04:05] Speaker 1: But your heart was giving you trouble, no? No. [00:04:07] Speaker 2: Well, that's something different, but you know, emergency escape if we need those. Now, what they decided to use was to use natural water. These are dry pipes filled with nitrogen, and so that if they have an alarm tripped, what they'll do is they'll flood this place out with water. Now, a couple things here. That sounds pretty darn scary, but this is a two-way confirmation system. Oh. When you pull the alarm, that is stage one, but just like launching a missile, right? You need two keys. We also have detectors that have to sense that there's smoke. Oh, the VESA detector that's on the wall. [00:04:38] Speaker 1: Absolutely. [00:04:39] Speaker 2: Okay. So you got to sense smoke, you got to sense heat rise, and then that's going to activate the system. So just one key is not going to launch the system, if you will. Very good. So that's a really nice way to do this. Number two, water doesn't affect data. Yes, it shorts out electronics. Yes, it shuts it down, but HALON, FM2, that damages electronics too. [00:04:56] Speaker 1: Which puts people in danger, raises your cost of all the emergency things you have to have. Absolutely. This is a large place to have all those respirators sitting around, right? Absolutely. [00:05:04] Speaker 2: And you got to maintenance that, you got to train them, you got to keep them up. Realistically, it's cheaper and more efficient to just use water. [00:05:09] Speaker 1: So final point in here, real quick, looking at the ceiling again. I notice, not only have they cabled, it looks like things that aren't even, they've terminated all the way down, even if there's not equipment in the racks yet. So they're ready for growth here. But the electric cabling, looks like the busway systems, they're dropping in a higher voltage here as well, correct? [00:05:23] Speaker 2: Well, so that's the European voltage center, that's the 480 that we're dropping in here. And, you know, we could literally do a whole show on this and why that's a real advantage. The big piece is, is that it does two really important things for any type of data center administrator. It allows us to get rid of a stage one step down transformer. That's why in here-- [00:05:42] Speaker 1: Which you almost have on a per rack basis, right? [00:05:44] Speaker 2: Absolutely. Yeah, you don't hear any whine. You hear those data centers whining, right? You don't hear that in this data center because we're using 480. 480 is being dropped off on each one of these bus bars and we're dropping down to 30 amps because we're not using 208. We don't have to use 60 amps supply gear. You know, you actually cut your amperage in half, which means you also can decrease your cable size. Cutting your cable size by half, what it really means is you cut your cable cost by half. Yeah. You cut your cable weight by half. And copper weighs a lot. Yeah. So it's a really nice advantage actually going to that-- [00:06:15] Speaker 1: And the conversion process generates heat, you waste, you don't get as much out as you did at the beginning of the process. Yeah. So, a lot of nice things from a green perspective as well there, right? [00:06:23] Speaker 2: Living in harmony, man. Living in harmony. [00:06:25] Speaker 1: All right. Can we take a look at the rack and kind of talk more specifically about-- [00:06:27] Speaker 2: I would want to show you some of the labeling stuff they did. That's pretty darn cool. All right. So let me show you, I think, one of the most impressive things to me. Now, this is the business end of a UCS cluster. Right. And I think that what I like the best is really the forethought that went into this, is that every single thing, every port that could be labeled or could be plugged in here is actually plugged in. That gives us the ability to really just wire this one time and then we could control and modulate our bandwidth, our applications, whatever remotely. We'll have to send somebody out to run another cable or anything like that. We actually have the ability to really control this remotely and truly have a real lights off data center if everything is cabled up. Now, another big piece and the thing I like about this entire complex, the thing I'm the most impressed about, is how unbelievably anal retentive they were about labeling everything. I mean, look at this one rack. Yeah. Out of the hundreds of racks in here, this is replicated all these times, is that everything in here is labeled and they're labeled with the correct wiring labels as well. This conforms to the PIA standard, they're zip-tied on both ends, they're very durable, so that every little piece in here, every power cord, even the fiber optics is labeled in here, which is pretty darn impressive. Now, I'll tell you something else that really gets me in here, is that not only are they labeled, everything is actually even RFID tagged as well. Yeah. So that now we can actually track your access points all over this place and we can track this stuff with RFID tags as well. [00:07:54] Speaker 1: Well, one of the key principles that was amazing to me is the fact that your RFID tagging down to the blade level, so at any given moment they know not only where everything is physically located, but they're measuring things down to the kilowatt load. So they know through these intelligent power strips, smart strips, you know, that exactly what are we pulling, what are we capable of pulling, so that you can really maximize utilization from end to end on the data center. And you know, are we balancing that, that magic act of when everything is full and fully utilized, did we use up all of our available power, can we fail over exactly what we want to fail over, and things of that nature. So the RFID tracking in there as well, I feel the heat coming off of here, so just, not to beat this point up too much, but we talked about the solid floors, we're pulling from the cold aisle on the other side, we've got these solid doors here, normally then this is going to flow up that chimney right there, and just naturally go back into that system for recooling the air back into the cold aisle, right? [00:08:43] Speaker 2: Yeah, and you mentioned the other side, let's walk around there and take a look at the more colorful end, and let me show you that side. [00:08:48] Speaker 1: All right. [00:08:49] Speaker 2: Obviously from the front of the cabinet, you can see that we've got a well-vented front door, so we can bring in that cool air, you can actually feel the equipment sucking that air in right now. You can go ahead and pop this open, you can actually see how we've carried this RFID asset tag tracking. Easy for you to say. Exactly, exactly. Down to each individual blade and chassis in here, so this is done as a forethought and not an afterthought. They're plugged in, we can track them right from the start, very, very cool. [00:09:19] Speaker 1: Well, so it's all about utilization, right? Yeah. So these guys can track it down to see how much we're using at any given moment for the really tying in the virtual and the physical, which becomes necessary to prevent some of that VM sprawl that we see a lot of. And then they've got a very flat network here as we go in, of course we've got out-of-band management, but then we've got unified fabric all the way back. [00:09:38] Speaker 2: Fabric extenders going back to those 7Ks, blanking panels, of course, good data for the design, that kind of stuff. [00:09:45] Speaker 1: So a lot of stuff to look at, let's close that sucker up because we have got a full agenda. So on to the rest of the data center, I think. You got it. Makes sense, yeah. One of the most surprising aspects here at the Allen Data Center that I would not have expected being a fellow Texan was the ability to use outside air to dramatically lower the cost of your energy usage for cooling air. Why do you need to cool it if it's already cooled? Well, here in this room, which is essentially an outdoor but inside space within the Allen Data Center, we've actually got open air access to the roof. We've got solar panels. We've got things covered so it's protected and such. But essentially, this allows air to be pulled in when the temperature is at a range that makes sense for the data center. Now, a couple of additional things have to be there to make this work. But essentially, we've got louvers here to pull the air in. We've got contaminant detectors so that it can be determined is the air safe to even bring in so that decisions can be made. But essentially, when the temperature gets down to a lower level, let's use the free air. It's a dramatic savings in cost and it's one of the more unique aspects here. [00:10:46] Speaker 2: Now, the fresh air from outside is actually coming in through the louvers and into the system itself. Now, if, for example, we actually detect any contaminants in the air or maybe a little too hot, we can close these louvers and open up these top louvers up here and bring in the air from the hot side of the data center. Either way, through the louvers or through the actual hot side louvers, this air comes in, it hits these paper filters. Now, this is a three-stage system, right? First stage, we're going to actually hit these paper filters, remove all the contaminants out of this. The second stage, it's going to hit the chiller, pull any heat out of it. And then the final stage is our humidifier stage. We'll actually make sure we're at the right humidity so this air comes out very conditioned, very clean, and certainly ready for data center use in this whole process. [00:11:32] Speaker 1: So I think this is what they call the air handler room, and it just seems to go on forever. [00:11:37] Speaker 2: Yeah. [00:11:38] Speaker 1: But this is somewhat the beginning of the process, right, in terms of whether you're bringing in on your closed loop system and you're recirculating air and kind of re-chilling it at each point back in the cycle again, or bringing in that free outside air. This is pretty unique. [00:11:50] Speaker 2: Yeah, this is actually pretty amazing, the work that went into planning and designing this stuff out. Because if you look at all this, this is one complete system. I mean, look at every one of these units are connected up to all this ductwork up here so that each one of these are part of a process. They're part of a cog in the machinery, if you will, that keeps the whole data center running. Not only that, but we also have alternative power sources here that we can use too. We've got an A and a B side that we can actually run these units on so we can switch power, modulate it, you know, do some load sharing if we want. And it really is pretty darn cool how we can really keep these data centers up and flowing and keep them breathing really well. [00:12:26] Speaker 1: Cool, you had to go for the cliché, didn't you? [00:12:28] Speaker 2: Yeah, absolutely, man. Especially if we're talking more of an open loop system, that's really cool stuff. [00:12:32] Speaker 1: Oh, you'd have to take it even further. Well, let me ask you though, when you look at this kind of capacity, and thankfully we're in kind of a ramp up mode here, the microphones are probably overwhelmed, I would think, from the noise that is potentially generated in the air handler room. What are we talking about here in terms of the amount of air that could be pushed when we're, you know, capacity down the road? [00:12:47] Speaker 2: Yeah, about a million and a quarter CFM. Wow. Yeah, it's pretty darn impressive stuff. [00:12:52] Speaker 1: Pretty darn impressive stuff. Wow, that is a lot of air permit. So this is obviously a major piece of the LEED certification of the notion of one of the elements that makes up a green data center. Right. Environmentally conscious about using all the natural mechanisms that are available to us, such as free outside air. [00:13:07] Speaker 2: Yeah, you know, this is the one thing about this data center that, you know me, I'm always hesitant to talk about green because it's usually a lot of politics involved. But this is one data center that really does it and gets it right. Because, you know, we're looking at, so in air handling, we're using fresh air to bring that in. The fresh air actually works in an open loop system. We're bringing it in from the outside, pushing it in, cleaning it up, of course, push out the data center, and then it vents out through the mezzanine deck on the fans that we have. Oh, because you're going to kick that outside. [00:13:33] Speaker 1: Why re-cool that? Why re-cool something? [00:13:35] Speaker 2: Okay. If we can access this good air, let's just kind of make the system open. If we have to close it and go on an internal air system, we're bringing the hot air in to cool that out. Then we bring that in, we go to more of a closed loop system, and it really does save the cache, and honestly really save the environment itself. It does add that nice green factor, so it's more than just talk. You know, a lot of people put solar panels on the roof, it's like, "Whee-hee, we're green." No, there's a lot of planning on here that really actually make this work like it's supposed to work, to be honest with you. [00:14:02] Speaker 1: Well, I think in the aspect of the closed loop system, it's taking a look at the chilled water and how that actually gets created, cleaned up, pumped through. Seems like a logical place to check next. [00:14:11] Speaker 2: I've got to show you that because there's some really cool advancements and some stuff they're doing here that really nobody else is doing in the industry, which I really like. [00:14:17] Speaker 1: So this is the chiller room? [00:14:19] Speaker 2: It's one of the chiller rooms, right. [00:14:20] Speaker 1: Oh, that makes sense, of course. So, with the different colored pipes and such, that's part of the labeling, right? Yeah, right. [00:14:26] Speaker 2: So, you know, you have different colored lagging actually shows. Not only you have the labels themselves that show which way the water flow is, but if those happen to fall off or whatever due to age, the lagging color shows you which direction this water flow goes. That water that goes down to, you know, one of our thousand-ton chillers here and actually pulls all the heat out of the chilled water that's coming from the data side. [00:14:45] Speaker 1: So, chilling water to cool the air, pretty basic, right? This data center's got tricks up its sleeve. What is the, that can't be the whole story here. It's really not. [00:14:53] Speaker 2: The cool stuff is actually what we're doing with the condenser water. Let's go take a look. Now, the condenser water itself is, in my opinion, the workhorse of the whole system, right? Right. Because this is what's actually grabbing the heat out of the chiller water and actually dissipating it as it goes out, hits our towers and stuff, and it gets all evaporated out. But now, the cool thing about this is, is that as this condenser water's coming through the system, is we're purifying this, and we're doing this with this dolphin up here. Now, this dolphin-- [00:15:22] Speaker 1: There's not a live dolphin in there. This is a dolphin brand. Dolphin brand, right. [00:15:25] Speaker 2: Okay. Right. So, one of the-- That's amazing. How'd you encapsulate that? One of the really cool things about this is, is that it's a proprietary method of how they purify the water, and they grab the bacteria and literally encapsulate it and starve it. So, that the bacteria can't-- So, you're talking about a chemical-free system. [00:15:42] Speaker 1: Yeah, yeah, yeah. Yeah, that's exactly what it is. [00:15:44] Speaker 2: Okay. And that's a real advantage that it gives, is that actually all this stuff, you know, it kills off all this bacteria without having to do any chemicals or any of that stuff, so that we can take this gray water, send it out to our pond, and that pond gets out there, and then it can actually fertilize and water all our plants and our landscaping and stuff. [00:16:00] Speaker 1: So, this makes a lot of sense, because one of the themes that we've kind of seen throughout here is all the different ways in which everything has been designed to work harmoniously with somewhat the way Mother Nature intended it, right? Yeah, right, right. Which is kind of one of the, let's ease into green from a perspective. It's not always just about power transformers and things of this nature. It's about simple things like this, so that we could use that for irrigation and not use other water, so we don't return chemicals, if you will, into the sewer system. That's right. And, you know, and who knows where that goes and what happens with this. That's right. They've even taken that extra step of working with indigenous plants designed to grow in this particular area, so that everything in the ecosystem that the data center represents is, again, harmonious, working within the flow. [00:16:38] Speaker 2: That's really cool. I like it. [00:16:40] Speaker 1: That's definitely cool. So, no stone unturned in this harmonious feature. [00:16:42] Speaker 2: Not a single one. [00:16:46] Speaker 1: So, here is the diesel tanks that are backing up those generators. If they go down and they need to run for an extended period of time, it's those diesel engines that kick in. But what's the genius behind that flywheel generator? Because it's dynamic rotor UPS. This is what makes it unique, right? [00:17:03] Speaker 2: Right. You know, that's a really cool part of this whole system, because typically on any data center out there, anywhere, anywhere in the world, anywhere in the country, when your voltage just comes in, you have some ripples in it, right? And that is really bad news for ones and zeros out there. So, what we want to do is flatline and smooth that out as much as possible. Now, what most data centers do is they have batteries, and those batteries will just eject and kind of keep that voltage at a certain threshold that we set for that data center. Now, the problem with that is that on batteries is that the more you do that, the weaker and weaker they get. Batteries have memory, and you're in this constant cycle of replacing these batteries all the time. [00:17:37] Speaker 1: Well, thinking about the green perspective, when you're replacing batteries, you have to dispose of your batteries. [00:17:41] Speaker 2: Yeah, now, we've got some really good recycling programs for batteries now, for sure. True. So, but still, you've got all that maintenance, you've got all that work, and that storage, you've got to store those things up. It's a real pain to tell for any data center admin. The really cool and genius part that they thought of here is a kind of an old-school concept of using a flywheel. Now, flywheels are things that we use like in clutches and cars and stuff like that. Right. But the advantage that they have is they're big and they're heavy, and as they spin around, they'll store energy. They'll want to keep moving continuously, right? [00:18:10] Speaker 1: So, that noise that we're hearing is really, you're running off normal power coming in through either the ground, either the A or the B feeds, right? Right. And it's driving that flywheel continuously at about, what, 3,000 RPM? [00:18:20] Speaker 2: About 3,000 RPM, yeah, on the outer, right? Because you have inner outer speeds and stuff, right? [00:18:24] Speaker 1: You've got two rings. You've got AC/DC, in fact, on those. That's correct. Creating the kinetic energy is how that's spinning it off from there. [00:18:29] Speaker 2: That's absolutely correct. Now, but the advantage that that has is that now we're using two different concepts here. We're using laws of inertia that if we have those ripples in there, then that flywheel continues to spin. It still has all that energy stored. So, it continues to spin around. It smooths them all out. And it actually levels that stuff out. If, however, that we don't, let's say we drop power, then that flywheel actually will now spin up. We can grab it with that electromagnetic clutch and engage it and start the diesel engines also if the batteries don't start the motor from that point. So, that's kind of a really cool concept. So, you're actually smoothing it out using the good old laws of inductance to actually grab and start your generators up. And then it's actually a secondary starting mechanism for that generator, too, which is pretty darn groovy, man. But, now, so, you know, we're actually out here. You can see the water tanks and things of that nature. Another really cool piece of that is is that typically in every data center, you're going to see they're going to want to remove heat. It's a motor. The motor's got that stuff. So, you know, you'll see these big fans sitting in front of there. There's a big radiators and stuff. We actually got all that stuff being heated and chillers are actually pulling all that heat out of that data center to a lot more efficient, you know, kind of water transfer method. You don't have to worry about humidity, right? [00:19:37] Speaker 1: Well, that's the nice thing about that motor because now you can use evaporative cooling. Exactly. Much lower cost form of cooling for that because it's got a wider range of conditions it can exist in, correct? Yeah, yeah, yeah. [00:19:46] Speaker 2: Very cool. So, you got cost savings. [00:19:48] Speaker 1: You don't have to eliminate unnecessary elements of energy that you've built up because it's always, you know, how can you keep from working against yourself? Right. And then that particular system there can actually replicate everything we're getting from the 10 megawatt feeds that we've got getting in. We can run completely off diesel generators should we have to here. These tanks are going to supply it for several days as I understand it. Right. And then when those tanks run low, of course, you've got service level agreements with tankers that can come in through these big doors and kind of keep this supply running. Same type of redundancy is happening not only for the power but we've got it for the water. We've got it for the coolers and the ability to route in between for anything you need there, right? [00:20:21] Speaker 2: Yeah. One of the guys was telling me that we're talking about how this whole flywheel, that these flywheels are machined so tight that if you turn those things off, the bearings are so precise that they'll continue to spin for a couple of hours. Oh, my gosh. Yeah, that's pretty darn amazing. [00:20:35] Speaker 1: Don't stop it with your hands. Very, very dangerous. You know, and one thing I want to make sure we point out here is that this entire facility that we've had the good fortune of being able to walk through in this kind of detail and share it is something that's been built for customers to come visit. So it's literally as simple as reaching out to your account manager. Oh, that's a great call, right. And this is literally, this is a briefing facility so customers can set appointments to their account managers, come and tour it and get hands-on exposure to kind of what we internally call that Cisco on Cisco story. [00:21:00] Speaker 2: Yeah, but out of every place in Cisco to see, besides TechWise TV set, this is the one place you definitely want to make sure you come to. [00:21:07] Speaker 1: This is one of the best data centers you've ever visited, I think. [00:21:09] Speaker 2: Beyond a shadow of a doubt, I've been to hundreds of data centers and I can tell you unquestionably, this is the most impressive, best built, most thought out data center I've ever seen ever. I mean, we're just, man, we're just touching like just 1% of all the cool stuff they've done here. Man, get a tour and come and see this place. You're not going to believe it. [00:21:27] Speaker 1: Very good point. [00:21:49] Speaker ?: Thank you.