Manual build

http://confluence.rcs.griffith.edu.au:8080/display/GHPC/python-lapack-atlas-sparse-numpy-scipy-matplotlib

RPM install is not the preferred method on this cluster but is available for rhel6.1

rpm install

Install on Admin

 yum install numpy
Loaded plugins: refresh-packagekit, rhnplugin
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package numpy.x86_64 0:1.3.0-6.2.el6 will be installed
--> Processing Dependency: python-nose for package: numpy-1.3.0-6.2.el6.x86_64
--> Processing Dependency: liblapack.so.3()(64bit) for package: numpy-1.3.0-6.2.el6.x86_64
--> Processing Dependency: libatlas.so.3()(64bit) for package: numpy-1.3.0-6.2.el6.x86_64
--> Processing Dependency: libptf77blas.so.3()(64bit) for package: numpy-1.3.0-6.2.el6.x86_64
--> Processing Dependency: libptcblas.so.3()(64bit) for package: numpy-1.3.0-6.2.el6.x86_64
--> Running transaction check
---> Package atlas.x86_64 0:3.8.3-12.4.el6 will be installed
---> Package python-nose.noarch 0:0.10.4-3.1.el6 will be installed
--> Processing Dependency: python-setuptools for package: python-nose-0.10.4-3.1.el6.noarch
--> Running transaction check
---> Package python-setuptools.noarch 0:0.6.10-3.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================================
 Package                                                   Arch                                           Version                                                   Repository                                                          Size
=============================================================================================================================================================================================================================================
Installing:
 numpy                                                     x86_64                                         1.3.0-6.2.el6                                             Red-Hat-Enterprise-Linux-6                                         1.6 M
Installing for dependencies:
 atlas                                                     x86_64                                         3.8.3-12.4.el6                                            Red-Hat-Enterprise-Linux-6                                         2.8 M
 python-nose                                               noarch                                         0.10.4-3.1.el6                                            Red-Hat-Enterprise-Linux-6                                         220 k
 python-setuptools                                         noarch                                         0.6.10-3.el6                                              Red-Hat-Enterprise-Linux-6                                         336 k

Transaction Summary
=============================================================================================================================================================================================================================================
Install       4 Package(s)

Total download size: 5.0 M
Installed size: 19 M

RPM install on base image

mount --bind /proc/ /compute/proc/
mount --bind /dev /compute/dev
yum --installroot=/compute/ install numpy
umount /compute/dev
umount /compute/proc

Install on all nodes (n001-n023)

manual install

numpy will not work with pythton3.x. It will only work with python 2.6 - 2.8

Make sure python-devl rpm is installed.

special install procedure for images only:
==========================================

mount --bind /proc/ /compute/proc/
mount --bind /dev /compute/dev
yum --installroot=/compute/ install python-devel
umount /compute/dev
umount /compute/proc

numpy install

numpy-1.6.1rc1.tar.gz

http://numpy.scipy.org/

mkdir /tmp/numpy
cd /tmp/numpy
cp /tmp/numpy-1.6.1rc1.tar.gz .
tar -zxvf numpy-1.6.1rc1.tar.gz

*Read the  INSTALL.txt*
(Also, this will give the compiler options : python setup.py build --help-fcompiler)


cd /tmp/numpy/numpy-1.6.1rc1

module load ATLAS/3.8.3
module load  nose/1.0.0
python setup.py build --fcompiler=gnu95



(we will use gfortran since atlas/bas were compiled using gfortran)
The following is not needed as we are not installing in python3.1.4 location.

cd /tmp/numpy/numpy-1.6.1rc1

module load ATLAS/3.8.3
module load  nose/1.0.0
module load python/3.1.4
python setup.py build --fcompiler=gnu95  install --prefix=/sw/python/3.1.4/


(python setup.py build --fcompiler=gnu95  install --prefix=/tmp/numpy/numpy-1.6.1)



The installation prefix (-prefix) is required only if you want to install NumPy in a location different from your Python installation directory (e.g. you are using the default Python executable provided by your system but you do not have root permissions). In this case (only), you will have to fill the $PYTHONPATH environment variable to indicate where Python can find NumPy.
In the other cases, Python will install the module into its own installation directory, which is fine in particular when you are using a custom installation of Python. In this case you will also not have to augment the $PYTHONPATH since the module is installed in a standard location known by Python.

Fortran ABI mismatch
====================

The two most popular open source fortran compilers are g77 and gfortran.
Unfortunately, they are not ABI compatible, which means that concretely you
should avoid mixing libraries built with one with another. In particular, if
your blas/lapack/atlas is built with g77, you *must* use g77 when building
numpy and scipy; on the contrary, if your atlas is built with gfortran, you
*must* build numpy/scipy with gfortran.

Choosing the fortran compiler
-----------------------------

To build with g77:

    python setup.py build --fcompiler=gnu

To build with gfortran:

    python setup.py build --fcompiler=gnu95

How to check the ABI of blas/lapack/atlas
-----------------------------------------

One relatively simple and reliable way to check for the compiler used to build
a library is to use ldd on the library. If libg2c.so is a dependency, this
means that g77 has been used. If libgfortran.so is a a dependency, gfortran has
been used. If both are dependencies, this means both have been used, which is
almost always a very bad idea.

In other words, for standard users, NumPy has a linear algebra module (linalg) which is 
linked with the system's blas/lapack/atlas libraries. This implies that the 
Fortran sources of NumPy must be compiled with the same compiler which was
used for the libraries. 
Check e.g. with ldd /usr/lib(64)/lapack.so which compiler you will have to use. 

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[root@admin ATLAS]# ldd 3.8.3/install/ATLAS/build/lib/libatlas.so
        linux-vdso.so.1 =>  (0x00007ffff7ffe000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ffff7465000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffff7247000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ffff6fc3000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003743000000)
[root@admin ATLAS]# ldd 3.8.3/install/ATLAS/build/lib/libcblas.so
        statically linked
[root@admin ATLAS]# ldd 3.8.3/install/ATLAS/build/lib/libf77blas.so
        linux-vdso.so.1 =>  (0x00007ffff7ffe000)
        libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007ffff7ad7000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ffff7852000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ffff74c1000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003743000000)
[root@admin ATLAS]# ldd 3.8.3/install/ATLAS/build/lib/liblapack.so
        linux-vdso.so.1 =>  (0x00007ffff7ffe000)
        libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007ffff7ad4000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ffff784f000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ffff74be000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003743000000)
[root@admin ATLAS]# ldd 3.8.3/install/ATLAS/build/lib/libptcblas.so
        statically linked
[root@admin ATLAS]# ldd 3.8.3/install/ATLAS/build/lib/libptf77blas.so
        linux-vdso.so.1 =>  (0x00007ffff7ffe000)
        libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007ffff7ad7000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ffff7852000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ffff74c1000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003743000000)

>>>>>>>>>>>>>>>>>>>>>>>>


Building with ATLAS support
=========================

usage

 module load  python/2.7.1

 python
Python 2.6.6 (r266:84292, Apr 11 2011, 15:50:32)
[GCC 4.4.4 20100726 (Red Hat 4.4.4-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
'1.3.0'
>>>

numpy with python 2.7.1-shared

module load python/2.7.1-shared

mkdir /tmp/numpy
cd /tmp/numpy
tar -zxvf numpy-1.6.1.tar.gz

create site.cfg in numpy directory
vi site.cfg

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[DEFAULT]
library_dirs = /sw/ATLAS/3.8.3/lib 
include_dirs = /sw/ATLAS/3.8.3/include/ 

[DEFAULT]
library_dirs = /sw/FFTW/3.2.2-gnu/lib 
include_dirs = /sw/FFTW/3.2.2-gnu/include 

[atlas]
atlas_libs = lapack, f77blas, cblas, atlas

[amd]
amd_libs = amd

[umfpack]
umfpack_libs = umfpack, gfortran

[fftw]
libraries = fftw3
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

python setup.py build --fcompiler=gnu95  install --prefix=/sw/python/2.7.1-shared/

numpy with python 3.1.4

module load python/3.1.4
Create site.cfg like above
python setup.py build --fcompiler=gnu95  install --prefix=/sw/python/3.1.4/