\(\mathit{\mathcal{N}etfiles}\) : An Enhanced Stream-Based Communication Mechanism
\(\mathit{\mathcal{N}etfiles}\) is an alternative API for message passing on distributed memory machines. Based on the communication stream model, \(\mathit{\mathcal{N}etfiles}\) provides enhanced functionality such as broadcasts and gather operations. \(
- PDF / 721,884 Bytes
- 8 Pages / 430 x 660 pts Page_size
- 38 Downloads / 193 Views
ract. Netfiles is an alternative API for message passing on distributed memory machines. Based on the communication stream model, Netfiles provides enhanced functionality such as broadcasts and gather operations. Netfiles overload conventional file I/O primitives enabling parallel programs to be developed and tested on a file system before execution on a parallel machine. Netfiles is part of a parallel programming system called FAbrIC. This paper also presents the design and implementation of the FAbrIC architecture and demonstrate the effectiveness of this approach by means of two parallel applications: a parallel shallow water model application and parallel Jacobi method. Keywords: Pipes, Stream Communication, File I/O, Message Passing, Parallel Programming.
1
Introduction
The stream communication model dates back to the early days of Unix. It encompasses mechanisms such as pipes, named pipes and sockets with TCP/IP. This model is simple, well-understood and has been successfully employed in many parallel programming systems. This paper presents an enhanced stream communication mechanism called Netfiles. The Netfile abstraction overloads file I/O primitives with message passing functionality specifically for parallel programming, with minimal deviation from semantics of conventional file I/O. Netfiles support point-to-point and collective communication. We have built an implementation of this abstraction and present an evaluation with two parallel applications.
2
Netfiles: Enhanced Pipes for IPC
In our proposed mechanism, communication is achieved through the reading and writing of a common Netfile, identified by a filename. This filename is a user-specified string that is task/node independent, allowing dynamic binding J. Labarta, K. Joe, and T. Sato (Eds.): ISHPC 2005 and ALPS 2006, LNCS 4759, pp. 254–261, 2008. c Springer-Verlag Berlin Heidelberg 2008
Netfiles: An Enhanced Stream-Based Communication Mechanism
255
Fig. 1. System architecture of FAbrIC
of process end-points. The sender opens the Netfile in WRITE mode, while the receiver opens in READ mode. When both ends are open, the Netfile effectively becomes a uni-directional communication channel. A write() behaves like a send(), and a read() behaves like a receive(). Non-blocking Netfile semantics is also possible, allowing communication to overlap with computation and potentially improving performance. No implicit type information is encoded within the stream. Thus, the data written through a Netfile do not have message boundaries, enhancing programming flexibility. For example, data sent via a write() may actually be read by several consecutive “small” read() operations. Collective operations are also supported in Netfiles. A single-writer, multiplereader Netfile behaves as a broadcast, closely resembling multiple reads on a single file. However, when readers share a single logical Netfile pointer, this operation behaves as a scatter. Similarly, a gather operation is represented as a single-reader Netfile with multiple writers appending data. Netfiles form part of a proposed
Data Loading...
