Agent-based Modeling with MASS CUDA on Multi-GPUs over Multi-Hosts

Abstract

Agent-based modeling (ABM) is an approach for simulating the behaviour of a system through simple interation between individual agents and the environment, which is widely adopted in various fields. For better simulation performance, attempts have been made to utilize the GPU to parallelize the simulation tasks, and the CUDA version of the Multi-Agent Spatial Simulation library (MASS CUDA) is one of the general purpose ABM libraries aims at providing high-level APIs for users to implement and execute their simulations on a single NVIDIA GPU. However, as the scale and complexity of the ABM simulation increase, a single GPU may not be sufficient to handle large-scale simulations that cannot fit into the device memory. To handle such situations, a new version of MASS CUDA is intorduced in this paper that support offloading simulation tasks across multiple GPUs over multiple hosts. With the buffer-based collaborative communication design and Nvidia Collective Communication Library (NCCL) for cross-device data transfer, collaboration among multiple GPUs on the same ABM simulation is realized. Together with the hybrid cross-hosts parallelism pattern, the GPUs can be distributed on different hosts. Moreover, with the refined memory layout for attributes and \codel|Place| and \codel|Agent| classes, the new version of MASS CUDA achieves significantly better performance and spatial scalability for large-scale simulations.