High Velocity Operating Systems Development

Abstract

Development velocity is critical to modern software development, particularly in thecloud, because it allows developers to ship features to users quickly. Despite Linux being the most commonly used cloud operating system, Linux development velocity is lagging, limiting its ability to adapt to new workloads and new hardware. High development velocity in operating systems is difficult due to the complexity of the code, the ease of introducing bugs, the difficulty of finding and fixing these bugs, and the disruption caused by kernel upgrades. Improving kernel development velocity introduces challenges not fully addressed by past research: limiting bugs and improving the speed of deployment and debugging without sacrificing generality, compatibility, or performance. Past efforts to improve safety by eliminating or isolating bugs has often come at a significant cost in terms of those requirements. We study bugs found in Linux to understand their causes and to suggest potential methods to reduce kernel bugs. We discover that around half of all security critical bugs are low level bugs that do not depend on the module logic. These could be prevented using language level compile-time safety checks, such as those performed by safe Rust. In this thesis, we show that it is possible to improve development velocity in commercial operating systems. We use Rust’s compile time safety checks to eliminate bugs without performance overhead or lack of generality. We also enable live upgrade and userspace debugging of kernel modules. We design, implement and evaluate Bento and Enoki, frame- works for improved development velocity in the Linux kernel. Bento is a framework for high velocity Linux kernel file systems that enables safe Rust, userspace debugging, and live upgrade. A file system implemented with Bento achieves performance comparable to Linux’s default file system and can be upgraded with only 15ms of service downtime. Enoki enables high velocity Linux kernel schedulers written in safe Rust. It supports record and replay debugging, live upgrade, and bidirectional userspace communication. Enoki sup- ports a variety of schedulers with performance on par with baseline Linux and research schedulers.