Distributed FaaSRunner: Enabling Reproducible Multi-node, Multi-threaded Function-as-a-Service Endpoint Testing

Abstract

Today's cloud native applications are often built using a service-oriented architecture supported by many microservices hosted using serverless Function-as-a-Service (FaaS) platforms. The vast majority of serverless function benchmarks and tests are performed using a single-client machine to generate workloads against highly scalable serverless backends. However, single-client test engines can lack sufficient computational resources and network bandwidth to adequately stress serverless backends. Testing tools such as Apache JMeter support orchestrating tests using multiple client nodes, but lack the ability to orchestrate sophisticated tests with distributed workload patterns common with real-world serverless workloads. In this thesis, we introduce Distributed FaaSRunner, a distributed test tool which supports the ability to reproduce multi-node serverless workloads using traces derived by ingesting serverless function log files and randomly generated workload traces. We test Distributed FaaSRunner's ability to precisely reproduce serverless function request dispatch and arrival time using various test cluster configurations. Using globally distributed clients, we predict latency to adjust workload trace event dispatch times to reproduce original request arrival latency. We demonstrate that Distributed FaaSRunner can reproduce both temporal and spatial characteristics of serverless workloads, enabling new capabilities to assess performance of FaaS platforms beyond traditional load testing.