mpi4py

http://code.google.com/p/mpi4py/

  • To build and install this package, you must meet the following
    requirements.

+ A Python 2.3/2.4/2.5/2.6/2.7/3.0/3.1/3.2 distribution.

+ A working MPI 1.2/1.3/2.0/2.1/2.2 implementation.

MPI for Python (mpi4py) provides bindings of the Message Passing Interface (MPI) standard for the Python programming language, allowing any Python program to exploit multiple processors.

This package is constructed on top of the MPI-1/MPI-2 specification and provides an object oriented interface which closely follows MPI-2 C++ bindings. It supports point-to-point (sends, receives) and collective (broadcasts, scatters, gathers) communications of any picklable Python object as well as optimized communications of Python object exposing the single-segment buffer interface (NumPy arrays, builtin bytes/string/array objects).

Installation notes

Build instructions in docs/source/usrman/install.rst

mkdir -p /sw/mpi/mpi4py/1.2.2
cd /sw/mpi/mpi4py/
tar -xvf /data1/mpi4py-1.2.2.tar.gz

If you already have a working MPI (either if you installed it from
sources or by using a pre-built package from your favourite GNU/Linux
distribution) and the :program:`mpicc` compiler wrapper is on your
search path, you can use :program:`pip`::

    $ [sudo] pip install mpi4py

or alternatively *setuptools* :program:`easy_install` (deprecated)::

    $ [sudo] easy_install mpi4py


 pip install mpi4py
Downloading/unpacking mpi4py
  Running setup.py egg_info for package mpi4py

Installing collected packages: mpi4py
  Running setup.py install for mpi4py
    MPI C compiler:    /sw/openMPI/1.4.3-gnu/bin/mpicc
    MPI C++ compiler:  /sw/openMPI/1.4.3-gnu/bin/mpicxx
    MPI linker:        /sw/openMPI/1.4.3-gnu/bin/mpicc
    checking for MPI compile and link ...
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c _configtest.c -o _configtest.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc _configtest.o -o _configtest
    success!
    removing: _configtest.c _configtest.o _configtest
    building 'mpi4py.MPI' extension
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c src/MPI.c -o build/temp.linux-x86_64-2.7/src/MPI.o
^COperation cancelled by user
Storing complete log in /root/.pip/pip.log
[root@admin mpi4py-1.2.2]# pip install mpi4py 2>&1 |tee mpi4pyBuildLog-aug19th2011.txt
Downloading/unpacking mpi4py
  Running setup.py egg_info for package mpi4py

Installing collected packages: mpi4py
  Running setup.py install for mpi4py
    MPI C compiler:    /sw/openMPI/1.4.3-gnu/bin/mpicc
    MPI C++ compiler:  /sw/openMPI/1.4.3-gnu/bin/mpicxx
    MPI linker:        /sw/openMPI/1.4.3-gnu/bin/mpicc
    checking for MPI compile and link ...
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c _configtest.c -o _configtest.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc _configtest.o -o _configtest
    success!
    removing: _configtest.c _configtest.o _configtest
    building 'mpi4py.MPI' extension
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c src/MPI.c -o build/temp.linux-x86_64-2.7/src/MPI.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc -shared build/temp.linux-x86_64-2.7/src/MPI.o -o build/lib.linux-x86_64-2.7/mpi4py/MPI.so
    checking for MPE availability ...
    checking for header 'mpe.h' ...
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c _configtest.c -o _configtest.o
    _configtest.c:3:17: error: mpe.h: No such file or directory
    failure.
    removing: _configtest.c _configtest.o
    failure.
    building 'mpi4py.MPE' extension
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c src/MPE.c -o build/temp.linux-x86_64-2.7/src/MPE.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc -shared build/temp.linux-x86_64-2.7/src/MPE.o -o build/lib.linux-x86_64-2.7/mpi4py/MPE.so
    checking for dlopen availability ...
    checking for header 'dlfcn.h' ...
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c _configtest.c -o _configtest.o
    success!
    removing: _configtest.c _configtest.o
    succes!
    checking for library 'dl' ...
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c _configtest.c -o _configtest.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc _configtest.o -ldl -o _configtest
    success!
    removing: _configtest.c _configtest.o _configtest
    building 'mpi4py.dl' extension
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DHAVE_DLFCN_H=1 -I/sw/python/2.7.1/include/python2.7 -c src/dynload.c -o build/temp.linux-x86_64-2.7/src/dynload.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc -shared build/temp.linux-x86_64-2.7/src/dynload.o -ldl -o build/lib.linux-x86_64-2.7/mpi4py/dl.so
    MPI C compiler:    /sw/openMPI/1.4.3-gnu/bin/mpicc
    MPI C++ compiler:  /sw/openMPI/1.4.3-gnu/bin/mpicxx
    MPI linker:        /sw/openMPI/1.4.3-gnu/bin/mpicc
    building 'python2.7-mpi' executable
    /sw/openMPI/1.4.3-gnu/bin/mpicc -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/sw/python/2.7.1/include/python2.7 -c src/python.c -o build/temp.linux-x86_64-2.7/src/python.o
    /sw/openMPI/1.4.3-gnu/bin/mpicc -Wl,-export-dynamic build/temp.linux-x86_64-2.7/src/python.o -L/usr/local/src/apps/lib -L/usr/local/src/apps/lib/python2.7/config -lpython2.7 -o build/exe.linux-x86_64-2.7/python2.7-mpi -lpthread -ldl -lutil -lm
    /usr/bin/ld: cannot find -lpython2.7
    collect2: ld returned 1 exit status
    warning: build_exe: building extension "python2.7-mpi" failed

    warning: build_exe: command '/sw/openMPI/1.4.3-gnu/bin/mpicc' failed with exit status 1

    /sw/python/2.7.1/bin/python -O /tmp/tmpIfxL2G.py
    removing /tmp/tmpIfxL2G.py

Successfully installed mpi4py
Cleaning up...

test

 Issuing at the command line::

    $ mpiexec -n 5 python demo/helloworld.py

or (in the case of older MPI-1 implementations)::

    $ mpirun -np 5 python demo/helloworld.py

will launch a five-process run of the Python interpreter and run the
test scripts :file:`demo/helloworld.py`.


You can also run all the *unittest* scripts::

    $ mpiexec -n 5 python test/runalltest.py

or, if you have nose_ unit testing framework installed::

    $ mpiexec -n 5 nosetests -w test

Results

h2. mpirun

 mpirun -np 5 python demo/helloworld.py
Hello, World! I am process 0 of 5 on admin.default.domain.
Hello, World! I am process 1 of 5 on admin.default.domain.
Hello, World! I am process 2 of 5 on admin.default.domain.
Hello, World! I am process 4 of 5 on admin.default.domain.
Hello, World! I am process 3 of 5 on admin.default.domain.