Intel Architecture Day 2021: Data Center - Infrastructure Processing Unit

[00:00:00] Speaker 1: Thanks, Roger. And this is exactly right. We are the middle of a revolution. So when we started this revolution, the systems that made up cloud data centers look pretty much like the systems in a classic enterprise data center. But that has changed. We're starting to see these two architectures diverge. And the reason for this is that in a classic data center, everything is owned by one party. In the cloud, the workload and the system owned by different ones, the tenant and the cloud service provider. So here's an example of a typical server in a classic enterprise data center. The physical infrastructure, the hypervisor and the application are all owned by one entity. In this case, it's a bank. All the software runs on the CPU. But for servers that are built for cloud infrastructure, where different architecture has emerged. They have a dedicated processor that runs the infrastructure functions in the cloud. And we call this new category of processor an IPU or Infrastructure Processing Unit. The cloud service provider software runs on this IPU and the revenue generating guest software runs on the CPU. So for example, a bank's financial app running on the CPU would not be cleanly separated from the cloud service providers infrastructure software running on the IPU. You know, if you want to think about an analogy, this is a little bit like hotels versus single family homes. In my home, I want it to be easy to move around from the living room to the kitchen to the dinner table. In a hotel, it's very different. The guest rooms and the dining hall and the kitchen are cleanly separated. The areas where the hotel staff works is different from the area where the hotel guests are. You need to get a badge if you want to move from one to the other in some cases. And essentially, this is the same trend that we're seeing in cloud infrastructure today. Now, this IPU based architecture has several major advantages. First, the strong separation of infrastructure functions and tenant workloads allows tenants to take full control of the CPU. Second, the cloud operator can offload infrastructure tasks to the IPU. This helps maximize the utilization of the CPU and for public clouds, also helps maximize revenue. And third, IPUs allow for fully diskless server architecture in the cloud data center. So let me explain each of these in more detail. So in servers with an IPU, infrastructure and tenant workloads are cleanly separated with the tenant workload running on the CPU and the infrastructure software running on the IPU. The media result of this is much better isolation between the two. So for example, if I have a spike in infrastructure load, it will no longer lead to performance issues for the CPU. That's obviously a very good property. But more importantly, it now allows the tenant to take full control of the CPU. So for example, a tenant can bring their own hypervisor and run it on the CPU. But at the same time, the IPU can still confine that hypervisor to a virtual network segment or specific storage volumes. That allows for much, much more flexible architecture. The second advantage of the IPU is about infrastructure function offload. So modern applications today are often structured as microservices that incur substantial communication overhead. In some cases, the majority of all CPU cycles are actually spent on the infrastructure overhead. And the IPU can help reduce this, as you can see in this slide here. With an IPU, the cloud operator can offload these infrastructure tasks to the IPU. And thanks to the IPU's accelerators, they can process these very, very efficiently. This not only optimizes performance, but if you're a cloud operator, you can now take 100% of the CPU cycles off that CPU and rent them out for guests, which helps to maximize your revenue for the overall system. The third advantage of the IPU is that it can enable the migration to a fully diskless server architecture. This is a big architectural change. And let me explain why this is a great thing. So traditionally, in a cloud data center, you will have disks attached to every single server. As tenant demand for disk space is hard to predict, you have to over-provision each of these servers, basically attach more disks than you really need, and you end up with a lot of stranded capacity. So capacity that can't be utilized in a good way. With an IPU, you can move to an entirely diskless model. As all storage is on a standalone storage service, and when a customer starts a workload on the server, the CSP basically creates virtual volume on the storage service. Via the management network, the CSP tells the IPU to create a new NVMe SSD based on that virtual volume. And as this virtual NVMe SSD shows up on the PCI Express first, just like a regular SSD, this will work with most operating systems and hypervisors out of the box, and we can now boot from that SSD. Now, you may wonder what does it do for performance. I mean, all this network traffic that is, you know, coming from these disks. And the really brilliant thing about the IPU here is that the actual storage traffic between the storage server and the workload on the server happens on the fast path, meaning there's no involvement of any CPU cores on the IPU or the CPU. It's, you know, it's low latency, it's high throughput with maximum flexibility, a very powerful solution. So with a strong separation of infrastructure and tenant, with accelerators that allow us to efficiently offload infrastructure functions, and this ability to move to a really diskless architecture, we think the IPU will be a central component for future data center architectures. Now, if you look at IPUs today, there's basically two types of architectures that are commonly used. The first one are dedicated ASIC IPUs, and the second one are FPGA-based IPUs. So each type has their own advantages and disadvantages. FPGA-based IPUs give you the ability to implement new protocols quickly. You can, you know, react to changing requirements or new protocols, or, you know, you can, for example, implement your proprietary protocols that are not publicly known on these FPGAs. On the other hand, a dedicated ASIC IPU maximizes performance and efficiency. And both of these are actually different from classic smart NICs, right, which lack the capability of executing the infrastructure control plane. Because there's no one-size-fit-all for, you know, the different types of infrastructure acceleration, Intel will continue to invest in both types of IPUs as well as smart NICs. We're deeply engaged with the world's leading cloud providers, including Microsoft, Baidu, JD.com, and VMware. And we are already the volume leader in the IPU market with our Z on D FPGA and Ethernet components. I'm thrilled to announce the arrival of two exciting new FPGA-based products in our IPU portfolio targeted for the cloud and comms market. And that's Oak Springs Canyon and Arrow Creek. Let's start with Oak Springs Canyon. Oak Spring Canyon is an FPGA-based IPU that uses Intel's Agilex FPGA together with the Xeon D system on a chip. Agilex is the industry's leading FPGA in power, efficiency, and performance, working in concert with Xeon-based servers to provide the performance necessary to offload two times hundred gig workloads and a rich software ecosystem optimized around x86. Oak Springs Canyon leverages the Intel Open FPGA stack, a scalable source accessible software and hardware infrastructure stack. Oak Springs Canyon is aligned with the needs for the next wave CSP deployments at 100 gig. Oak Springs Canyon also features a hardened crypto block that allows you to secure all infrastructure traffic, storage, and networking at line rate performance. And today that's an important thing. So the second product I want to talk about today is called Arrow Creek. Arrow Creek is an acceleration development platform based on the Agilex FPGA and the E810 100 gig ethernet controller. It builds upon the success of Intel's PAC M3000 which is deployed today at some of the top comm service providers worldwide. Arrow Creek will help telco providers to offer flexible accelerated workloads like Juniper Contrails, OVS, and SRV6. With these two FPGA-based additions to our portfolio, Intel covers the needs of both cloud and communications service providers. What I'm actually most excited about today is that we're announcing Intel's first dedicated ESIC-based IPU, codename Mount Evans. Co-developed with a large CSP, Mount Evans is the foundation of a family of forthcoming ESIC IPUs. So, Naroo, you're one of the key architects of the amazing team that built this technology. Tell us more about it. Thank you, Guido. [00:08:31] Speaker 2: As Guido just mentioned, Intel is helping to lead this industry transformation by building leadership IPUs based on our FPGA and ASIC assets. I'm here today to introduce you to a product I'm really excited about. That product, codenamed Mount Evans, is our first 200 gig ASIC IPU or infrastructure processing unit. We have architected and developed Mount Evans hand-in-hand with a top cloud provider. This has provided tremendous insights into deployment requirements for networks at scale. Intel has been working closely with other cloud providers through our FPGA-based solutions and are learning for those products that influence many of the Mount Evans architecture and design trade-offs. Mount Evans has been designed for performance at scale under real-world workloads. Finally, in order to be hyperscale-ready, we designed in security and isolation from the ground up throughout the chip. On the technology front, Mount Evans is loaded with innovation. To start with, the focal point of the product is what we believe to be a best-in-class packet processing engine that supports a large number of existing use cases like vSwitch offload, firewalls, and virtual routing, as well as providing significant headroom for future use cases. Another technology, created by extending Intel's proven high-performance Optane NVMe controller, enables Mount Evans to emulate NVMe devices. A third technology innovation I'm excited about is a next-generation reliable transport protocol. We have co-innovated on this technology with our CSP partner to solve the long tail latency problem on lossy networks. Lastly, a fourth enabling technology that can be used across a variety of use cases is our advanced crypto and compression accelerators, leveraging our high-performance QuickAssist technology. Finally, at Intel, we really want to make IPUs a compelling technology across segments beyond cloud, and this, first and foremost, means enabling software developers to do what they do best. We start with innovative, performant hardware designed for flexibility and ease of programmability. We add to this the expertise that came in through our Barefoot acquisition, driving the use of the P4 language in the industry as a standard framework for programming network data planes onto IPUs. We'll extend well-known SDKs like DPDK and SPDK to take advantage of IPU capabilities for data and storage processing. Here, I'm showing a high-level block diagram of Mount Evans. As you can see, Mount Evans is organized as a networking subsystem on the left and a compute subsystem on the right. I won't go through every block in the short time we have today, but I did want to highlight a few areas. Mount Evans supports 200 gigabits per second of throughput, connecting up to four Xeon hosts together. We recognize that cloud performance needs will drive many applications like storage, messaging, and high-performance computing to migrate to RDMA-based protocols. Mount Evans supports this with implementations of both Rocky V2 and the new reliable transport technology I mentioned earlier. Our Optane-derived NVMe engine exposes high-performance NVMe devices to the host processors, enabling infrastructure providers to use the IPU to implement their storage protocol of choice, whether it's hardware accelerated NVMe over fabrics or a custom software backend on the compute system. The programmable packet processor delivers leadership support for use cases like vSwitch offload, firewalls, telemetry functions, all while supporting up to 200 million packets per second performance on real-world implementations. Finally, Mount Evans provides inline IP set to secure every packet being sent across the network. On the right-hand side, our compute complex is built on the ARM Neoverse architecture using the N1 Ares core. These 16 high-frequency cores come with a large system-level cache backed by three LPDDR4 controllers. The compute complex is tightly coupled with the network subsystem, allowing the network subsystem accelerators to use the system-level cache as a last-level cache, providing high bandwidth, low latency connections between the two, and enabling a flexible combination of hardware and software packet processing. Our look-aside crypto and compression engine is derived from Intel's QuickAssist technology that you can see in the Xeon roadmap, but we've adapted it for IPU use models. This includes support for the Z-Standard compression algorithm. Finally, our dual-core management processor provides an interface to the platform and orchestration layers supporting robust system manageability. We designed Mount Evans from a software-first mindset. Enabling applications and IPUs requires a robust software foundation. I already shared a few details on using the P4 language for programming network data planes and extending well-known SDKs like DPDK and SPDK. We'll share more details in the next few months. Thank you. Back to you, Raja.