Performance comparison of multi-container deployment schemes for HPC workloads: an empirical study
- PDF / 3,680,876 Bytes
- 40 Pages / 439.37 x 666.142 pts Page_size
- 95 Downloads / 186 Views
Performance comparison of multi‑container deployment schemes for HPC workloads: an empirical study Peini Liu1,2 · Jordi Guitart1,2 Accepted: 16 November 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract The high-performance computing (HPC) community has recently started to use containerization to obtain fast, customized, portable, flexible, and reproducible deployments of their workloads. Previous work showed that deploying an HPC workload into a single container can keep bare-metal performance. However, there is a lack of research on multi-container deployments that partition the processes belonging to each application into different containers. Partitioning HPC applications has shown to improve their performance on virtual machines by allowing to set affinity to a non-uniform memory access (NUMA) domain for each of them. Consequently, it is essential to understand the performance implications of distinct multi-container deployment schemes for HPC workloads, focusing on the impact of the container granularity and its combination with processor and memory affinity. This paper presents a systematic performance comparison and analysis of multi-container deployment schemes for HPC workloads on a single-node platform, which considers different containerization technologies (including Docker and Singularity), two different platform architectures (UMA and NUMA), and two application subscription modes (exact subscription and over-subscription). Our results indicate that finer-grained multi-container deployments, on the one side, can benefit the performance of some applications with low interprocess communication, especially in over-subscribed scenarios and when combined with affinity, but, on the other side, they can incur some performance degradation for communication-intensive applications when using containerization technologies that deploy isolated network namespaces. Keywords Docker · Singularity · Performance analysis · Deployment schemes · Multi-container · HPC workloads * Peini Liu [email protected] Jordi Guitart [email protected] 1
Computer Science Department, Barcelona Supercomputing Center (BSC), Barcelona, Spain
2
Computer Architecture Department, Universitat Politecnica de Catalunya (UPC), Barcelona, Spain
13
Vol.:(0123456789)
P. Liu, J. Guitart
1 Introduction Modern computing infrastructure is evolving at a fast pace from using dedicated physical datacenters to cloud computing services. Virtualization, as a fundamental technology for cloud computing, allows efficient utilization and easy maintenance of the infrastructure. So far, this attractive paradigm has been widely used by leading commercial companies and communities to manage their clusters [16, 32]. The HPC community is also involved in this transformation of adopting virtualization to benefit from some of its well-known advantages [49], such as the encapsulation of specific software environments for each user, which allows for customization, portability, and research reproducibility [19]; the isolat
Data Loading...
