Accelerating Networked Systems with Programmable and Tightly-Coupled NICs

Abstract

As datacenter networks evolve to support higher bandwidths and lower latency, delivering this performance to applications is an increasing challenge. Software packet-processing overheads and high-cost hardware data paths are main contributors to this challenge. Offloading packet-processing work from host CPU cores to the network interface controller (NIC) is one compelling solution. While existing hardware offloads are limited in flexibility, SmartNIC devices with an integrated system-on-chip (SoC) are fully programmable. However, achieving efficiency improvements using SmartNICs depends on specific architectural features, and efficient interfaces which only some devices provide. Additionally, SoC resources are inherently limited, and host-NIC communication still relies on high-overhead PCI Express transfers. This suggests that distributed systems must be carefully integrated with SmartNIC resources and interfaces, in order to benefit from these devices. Looking forward, new standards for cache-coherent interconnects have the potential to simplify host-NIC communication. These interconnects bring devices into the CPU's coherence domain, promising high performance and simpler sharing semantics. This thesis makes the argument for tight integration of datacenter systems and NICs at multiple levels, with the goal of increasing performance and efficiency. First, we conduct a measurement characterization of the SoC SmartNIC design space, with the goal of understanding opportunities for performance and efficiency gains. Second, we present Xenic, a case study of integrating SmartNIC resources with the design of a distributed transaction processing system. By considering the NIC resources and interfaces in its system design, Xenic accelerates performance and minimizes the use of high-cost data paths. Finally, we present CC-NIC, a NIC interface design optimized for emerging cache-coherent interconnects. CC-NIC demonstrates that integrating the NIC interface into the CPU's coherence domain offers the potential to improve throughput, latency, and CPU packet-handling efficiency.