| Cambridge-Cranfield HPCF > Information for Users > Programming and Compilation > MPI |
This page contains some elementary examples of the use of MPI on the CCHPCF. More general documentation on MPI can be found from the page of useful HPCF links.
MPI (Message Passing Interface) is a small library of routines concerned with explicit message passing on parallel computers. It has a well-defined standard and is not specific to one vendor. The portability of MPI is sufficiently high that there is no need to produce a lot of CCHPCF-specific documentation for it: any well-behaved MPI program should run immediately, and any general MPI course or book should be relevant for the CCHPCF.
Although the individual MPI routines do not have their own man pages on the CCHPCF, man mpi does provide an introduction.
There follow two trivial examples:
and finally a couple of comments on I/O under MPI.
program hello
c
include 'mpif.h'
integer npe,mype,ierr
c
call mpi_init(ierr)
if (ierr.ne.0) stop 'MPI initialisation error'
c
call mpi_comm_rank(mpi_comm_world, mype, ierr)
call mpi_comm_size(mpi_comm_world, npe, ierr)
c
write(*,101) mype,npe
101 format(' Hello parallel world, I am process ',I3,
& ' out of ',I3)
c
call mpi_finalize(ierr)
end
This could be compiled by typing (on franklin-1)
mpf90 -o hello.mpi -lmpi hello.f
and then run with a script containing
mpirun -np 4 ./hello.mpi
the output is
Hello parallel world, I am process 0 out of 4 Hello parallel world, I am process 3 out of 4 Hello parallel world, I am process 2 out of 4 Hello parallel world, I am process 1 out of 4
Note the same program is run on each of the 4 nodes in the partition. On each node calls to the MPI library are used to set the variables mype and npe to the rank number of the asking process (unique to each node) and to the total number of process (same for all nodes). These values are printed out in a single line. Because there are 4 active nodes 4 lines of output are produced. As all nodes are producing their line of output "simultaneously", the ordering of the lines in the output file will be random.
If using free format FORTRAN90, then the file 'mpif90.h' should be included, rather than 'mpif.h' which is in fixed format. Unfortunately this file does not include any prototypes, so no argument checking will occur, even under FORTRAN90.
There now follows the above example in C.
#include <stdio.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
int this_process;
int num_processes;
MPI_Status status;
int error_code;
error_code = MPI_Init(&argc,&argv);
if(error_code != MPI_SUCCESS){
printf("MPI initialisation error\n");
exit(1);
}
MPI_Comm_size(MPI_COMM_WORLD,&num_processes);
MPI_Comm_rank(MPI_COMM_WORLD,&this_process);
printf("Hello parallel world, I am process %d out of %d\n",
this_process,num_processes);
MPI_Finalize();
}
This could be compiled by typing (on a workstation)
mpcc -o hello hello.c -lmpi
and the rest of the story is the same as for FORTRAN. In general the FORTRAN and C MPI calls are similar, but the C calls tend to be integer functions returning their error code, whereas the FORTRAN calls tend to be subroutines taking an extra integer parameter which is used for the error code.
There are many approaches to I/O in parallel environments, and there are a few extra issues which must be considered.
Each processor runs its own copy of the program, so it can open and close files and maintain file pointers independently of the other nodes. When reading from a file, this is (generally) the desired behaviour: each node will read the whole of the file as though none of the other nodes existed. When writing, this (generally) leads to undesired behaviour - every node may write its first piece of output at offset zero from the start of the file, and the result becomes ill-defined.
There are two simple approaches to parallel I/O: either every node passes the data back to some "master" node which then does the writing, or every node writes to a separate file. These two models are the simplest, and should be sufficient for most purposes. A FORTRAN fragment demonstrating the latter technique could be:
character*20 filename
write(filename,123) mype
123 format('myfile',I3.3)
open(10,file=filename)
write(10,*)' Hello from ',mype
close(10)
Notice that each node can use the same unit number (10), because the programs running on each node are distinct at this level. However, each must specify a different filename.
If much data is to be read on all nodes, reading on one node and then broadcasting is generally more efficient than reading on all nodes simultaneously.
Users with large I/O requirements, or requirements which do not fit into the models above, may wish to contact us (support@hpcf.cam.ac.uk) for further advice.