MPI + MPI: a new hybrid approach to parallel programming with MPI plus shared memory
- PDF / 568,948 Bytes
- 16 Pages / 439.37 x 666.142 pts Page_size
- 105 Downloads / 267 Views
MPI + MPI: a new hybrid approach to parallel programming with MPI plus shared memory Torsten Hoefler · James Dinan · Darius Buntinas · Pavan Balaji · Brian Barrett · Ron Brightwell · William Gropp · Vivek Kale · Rajeev Thakur
Received: 22 December 2012 / Accepted: 25 April 2013 © Springer-Verlag Wien 2013
Abstract Hybrid parallel programming with the message passing interface (MPI) for internode communication in conjunction with a shared-memory programming model to manage intranode parallelism has become a dominant approach to scalable parallel programming. While this model provides a great deal of flexibility and performance potential, it saddles programmers with the complexity of utilizing two parallel
T. Hoefler (B) ETH Zurich, Zurich, Switzerland e-mail: [email protected] J. Dinan · D. Buntinas · P. Balaji · R. Thakur Argonne National Laboratory, Argonne, IL, USA e-mail: [email protected] D. Buntinas e-mail: [email protected] P. Balaji e-mail: [email protected] R. Thakur e-mail: [email protected] B. Barrett · R. Brightwell Sandia National Laboratories, Sandia, NM, USA e-mail: [email protected] R. Brightwell e-mail: [email protected] W. Gropp · V. Kale University of Illinois at Urbana-Champaign, Urbana, IL, USA e-mail: [email protected] V. Kale e-mail: [email protected]
123
T. Hoefler et al.
programming systems in the same application. We introduce an MPI-integrated sharedmemory programming model that is incorporated into MPI through a small extension to the one-sided communication interface. We discuss the integration of this interface with the MPI 3.0 one-sided semantics and describe solutions for providing portable and efficient data sharing, atomic operations, and memory consistency. We describe an implementation of the new interface in the MPICH2 and Open MPI implementations and demonstrate an average performance improvement of 40 % to the communication component of a five-point stencil solver. Keywords
MPI-3.0 · Shared memory · Hybrid parallel programming
Mathematics Subject Classification 68N19 other progamming techniques (objects-oriented, sequential, concurrent, automatic, etc.)
1 Introduction The message passing interface (MPI [1]) has been the dominant parallel programming model since the mid-1990s. One important reason for this dominance has been its ability to deliver portable performance on large, distributed-memory massively parallel processing (MPP) platforms, large symmetric multiprocessing (SMP) machines with shared memory, and hybrid systems with tightly coupled SMP nodes. For the majority of these systems, applications written with MPI were able to achieve acceptable performance and scalability. However, recent trends in commodity processors, memory, and networks have created the need for alternative approaches. The number of cores per chip in commodity processors is rapidly increasing, and memory capacity and network performance are not able to keep up the same pace. Because memory capacity per core is decreasing, mapping a single operating system process to an MPI rank an
Data Loading...
