BigDFT
- Indy Siva
Owned by Indy Siva
Dec 10, 2015
25 min read
Loading data...
BigDFT
http://inac.cea.fr/L_Sim/BigDFT/
BigDFT is a DFT massively parallel electronic structure code using a wavelet basis set. It was developed in an European program. Wavelets form a real space basis set distributed on an adaptive mesh (two levels of resolution in our implementation). GTH or HGH pseudopotentials are used to remove the core electrons. Thanks to our Poisson solver based on a Green function formalism, periodic systems, surfaces and isolated systems can be simulated with the proper boundary conditions.
Installation Instruction: http://inac.cea.fr/L_Sim/BigDFT/install.html
Usage
module load BigDFT/1.6.0-gnu-cuda-openmpi
Other versions are also available:
module load BigDFT/1.6.0-gnu OR module load BigDFT/1.6.0-gnu-cuda OR module load BigDFT/1.6.0-gnu-cuda-openmpi OR module load BigDFT/1.6.0-gnu-openmpi
module display BigDFT/1.6.0-gnu-cuda-openmpi
module display BigDFT/1.6.0-gnu-cuda-openmpi ------------------------------------------------------------------- /sw/com/modulefiles/BigDFT/1.6.0-gnu-cuda-openmpi: module-whatis Name: BigDFT module-whatis Version: 1.6.0 module-whatis Category: NAMD,Cuda module-whatis Examples: <BigDFT_base>/examples/fortran/add_array AND <BigDFT_base>/examples/c module-whatis url: http://inac.cea.fr/L_Sim/BigDFT/install.html module-whatis Description: BigDFT is a DFT massively parallel electronic structure code using a wavelet basis set module load cuda/4.0 module load opencl/cudasdk-4 module load opencl/opencl-1.1 module load mpi/openMPI/1.4.3-gnu module load NetCDF/4.1.3-gnu module load ATLAS/3.9.39 conflict intel-cmkl-11/11.1.072 conflict intel-mpi/4.0.0.027 conflict intel-tools-11/11.1.072 conflict intel-cc-11/11.1.072 conflict intel-fc-11/11.1.072 prepend-path PATH /sw/BigDFT/1.6.0-gnu-cuda-openmpi/bin prepend-path LD_LIBRARY_PATH /sw/BigDFT/1.6.0-gnu-cuda-openmpi/lib prepend-path LD_LIBRARY_PATH /sw/BigDFT/1.6.0-gnu-cuda-openmpi/lib/pkgconfig prepend-path INCLUDE /sw/BigDFT/1.6.0-gnu-cuda-openmpi/include prepend-path LDFLAGS -L/sw/BigDFT/1.6.0-gnu-cuda-openmpi/lib prepend-path LDFLAGS -L/sw/BigDFT/1.6.0-gnu-cuda-openmpi/lib/pkgconfig prepend-path CFLAGS -I/sw/BigDFT/1.6.0-gnu-cuda-openmpi/include prepend-path CPPFLAGS -I/sw/BigDFT/1.6.0-gnu-cuda-openmpi/include prepend-path CXXFLAGS -I/sw/BigDFT/1.6.0-gnu-cuda-openmpi/include prepend-path FCFLAGS -I/sw/BigDFT/1.6.0-gnu-cuda-openmpi/include prepend-path FFLAGS -I/sw/BigDFT/1.6.0-gnu-cuda-openmpi/include -------------------------------------------------------------------
pbs script templates
Sample pbs script
#!/bin/bash -l #PBS -m abe ### Mail to user #PBS -M <YourEmail>@griffith.edu.au #PBS -V ### Job name #PBS -N N2_test_bigDFT #PBS -q gpu #PBS -l walltime=12:00:00 ### Number of nodes:Number of CPUs:Number of threads per node #PBS -l select=1:ncpus=4:mem=4gb:mpiprocs=4:ngpus=1 ###$PBS_NODEFILE is a node-list file created with select and mpiprocs options by PBS ###### The number of MPI processes available is mpiprocs * nodes (=NPROCS) NPROCS=4 #PBS -j oe # This job's working directory echo Working directory is $PBS_O_WORKDIR cd $PBS_O_WORKDIR source $HOME/.bashrc module load BigDFT/1.6.0-gnu-cuda-openmpi #module list echo "Starting job" echo Running on host `hostname` echo Time is `date` echo Directory is `pwd` #echo This jobs runs on the following processors: #echo `cat $PBS_NODEFILE` /sw/openMPI/1.4.3-gnu/bin/mpirun -machinefile $PBS_NODEFILE -np $NPROCS env PATH=$PATH env LD_LIBRARY_PATH=$LD_LIBRARY_PATH bigdft echo "Done with job"
Sample input data
cat posinp.xyz
2 angstroem free N 0. 0. 0. N 0. 0. 1.11499
Interested in intereactive batching..Try this:
qsub -I -l select=1:ncpus=4:mem=4gb:mpiprocs=4:ngpus=1 -l walltime=01:00:00 -q gpu
special compile notes
Running configure caused a lot of issues. however, after running local versions of blas-3.2.1-4.el6.x86_64.rpm blas-devel-3.2.1-4.el6.x86_64.rpm lapack-3.2.1-4.el6.x86_64.rpm lapack-devel-3.2.1-4.el6.x86_64.rpm packages, the configure went well. lapack-3.2.1-4.el6.x86_64.rpm 100% 4422KB 4.3MB/s 00:00 blas-devel-3.2.1-4.el6.x86_64.rpm 100% 133KB 133.0KB/s 00:00 blas-3.2.1-4.el6.x86_64.rpm 100% 321KB 321.5KB/s 00:00 lapack-devel-3.2.1-4.el6.x86_64.rpm 100% 4571KB 4.5MB/s 00:00 Preparing... ################################################## blas ################################################## blas-devel ################################################## lapack ################################################## lapack-devel ##################################################
Versions
gnu versions
gnu version with Cuda
cd /sw/BigDFT/1.6.0-intel/src/bigdft-1.6.0-gnu
module load cuda/4.0
module load opencl/cudasdk-4
module load opencl/opencl-1.1
module load mpi/openMPI/1.4.3-gnu
module load NetCDF/4.1.3-gnu
module load ATLAS/3.9.39
./configure --enable-cuda-gpu --with-cuda-path=/usr/local/cuda --prefix=/sw/BigDFT/1.6.0-gnu-cuda CXX=c++ CC=gcc FC=gfortran F77=gfortran --with-netcdf-path=/sw/netCDF/4.1.3-gnu --disable-mpi 2>&1 |tee configure.gnu-cuda.txt
>>>>>>>>>>>>>>>>>>>>>>>
Basics:
C compiler: gcc
C flags:
CXX compiler: c++
CXX flags: -g -O2
Linker flags:
Linked libraries:
NVidia Cuda Compiler: /usr/local/cuda/bin//nvcc
NVCC flags: -g -O3 -DHAVE_CONFIG_H -I. -I.. -arch sm_13
NVCC C Compiler:
Cuda emulation: no
Trace functions: no
Verbose: no
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-cuda
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
BigDFT configure summary:
========================
Basics:
Fortran90 compiler: gfortran
Fortran90 compiler name:
Fortran90 flags: -g -O2
Fortran77 compiler: gfortran
Fortran77 flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64 -L$(top_builddir)/libxc-1.1.0/src/.libs -L$(top_builddir)/libABINIT/src
Linked libraries: $(top_builddir)/src/libbigdft.a -labinit $(top_builddir)/src/libdft_common.a -lxc $(top_builddir)/S_GPU/src/.libs/libs_gpu.a -lstdc++ -lcudart -lcublas -llapack -lblas -larchive -lrt
Build:
Library ABINIT: yes
Library BigDFT: yes
Main binaries (cluster...):yes
Minima hopping binary: no
atom and pseudo binaries: no
User documentation: yes
Devel. documentation: yes / no
Options:
Debug version: no
With MPI: no
| include dir.: -I$(top_builddir)/src/modules
| linker flags:
| linked libraries:
| MPI_INIT_THREAD avail:
| MPI2 support:
With optimised conv.: yes
With Cuda GPU conv.: yes
| NVidia Cuda Compiler: /usr/local/cuda/bin/nvcc
| Cuda flags:
With OpenCL support: no
With dgemmsy support: no
With libXC: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libxc-1.1.0/src
With libABINIT: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libABINIT/src
With libS_GPU: yes
| internal built: yes
| include dir.: -DCUBLAS_USE_THUNKING -I$(top_srcdir)/S_GPU/src
With ETSF_IO: no
| include dir.:
With libarchive: yes
| C flags:
| linked libraries: -larchive
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-cuda
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
Fortran modules: ${prefix}/include/
Documentation: ${datarootdir}/doc/${PACKAGE_TARNAME}
>>>>>>>>>>>>>>>>>>>>>>>>>>
make 2>&1 |tee make.gnu-cuda.txt
make check 2>&1 make-check.gnu-cuda.txt
make install 2>&1 |tee make-install.gnu-cuda.txt
gnu version with Cuda and openMPI
Make sure these packages are installed locally: blas-3.2.1-4.el6.x86_64.rpm blas-devel-3.2.1-4.el6.x86_64.rpm lapack-3.2.1-4.el6.x86_64.rpm lapack-devel-3.2.1-4.el6.x86_64.rpm
cd /sw/BigDFT/1.6.0-intel/src/bigdft-1.6.0-gnu-cuda-openmpi
module load cuda/4.0
module load opencl/cudasdk-4
module load opencl/opencl-1.1
module load mpi/openMPI/1.4.3-gnu
module load NetCDF/4.1.3-gnu
module load ATLAS/3.9.39
./configure --enable-cuda-gpu --with-cuda-path=/usr/local/cuda --prefix=/sw/BigDFT/1.6.0-gnu-cuda-openmpi CXX=mpic++ CC=mpicc FC=mpif90 F77=mpif77 --with-netcdf-path=/sw/netCDF/4.1.3-gnu 2>&1 |tee configure.gnu-cuda-openmpi.txt
(Not needed: --with-mpi-include=-I/sw/openMPI/1.4.3-gnu/include)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Basics:
C compiler: mpicc
C flags:
CXX compiler: mpic++
CXX flags: -g -O2
Linker flags: -L/sw/ATLAS/3.9.39/lib:-L/usr/local/cuda/lib64
Linked libraries:
NVidia Cuda Compiler: /usr/local/cuda/bin//nvcc
NVCC flags: -g -O3 -DHAVE_CONFIG_H -I. -I.. -arch sm_13
NVCC C Compiler:
Cuda emulation: no
Trace functions: no
Verbose: no
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-cuda-openmpi
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
BigDFT configure summary:
========================
Basics:
Fortran90 compiler: mpif90
Fortran90 compiler name:
Fortran90 flags: -g -O2
Fortran77 compiler: mpif77
Fortran77 flags: -g -O2
Linker flags: -L/sw/ATLAS/3.9.39/lib:-L/usr/local/cuda/lib64 -L/usr/local/cuda/lib64 -L$(top_builddir)/libxc-1.1.0/src/.libs -L$(top_builddir)/libABINIT/src
Linked libraries: $(top_builddir)/src/libbigdft.a -labinit $(top_builddir)/src/libdft_common.a -lxc $(top_builddir)/S_GPU/src/.libs/libs_gpu.a -lstdc++ -lcudart -lcublas -llapack -lblas -larchive -lrt
Build:
Library ABINIT: yes
Library BigDFT: yes
Main binaries (cluster...):yes
Minima hopping binary: no
atom and pseudo binaries: no
User documentation: yes
Devel. documentation: yes / no
Options:
Debug version: no
With MPI: yes
| include dir.:
| linker flags:
| linked libraries:
| MPI_INIT_THREAD avail: yes
| MPI2 support: yes
With optimised conv.: yes
With Cuda GPU conv.: yes
| NVidia Cuda Compiler: /usr/local/cuda/bin/nvcc
| Cuda flags:
With OpenCL support: no
With dgemmsy support: no
With libXC: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libxc-1.1.0/src
With libABINIT: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libABINIT/src
With libS_GPU: yes
| internal built: yes
| include dir.: -DCUBLAS_USE_THUNKING -I$(top_srcdir)/S_GPU/src
With ETSF_IO: no
| include dir.:
With libarchive: yes
| C flags:
| linked libraries: -larchive
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-cuda-openmpi
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
Fortran modules: ${prefix}/include/
Documentation: ${datarootdir}/doc/${PACKAGE_TARNAME}
>>>>>>>>>>>>>>>>>>>>>>>>
make 2>&1 |tee make.gnu-cuda-openmpi.txt
make check 2>&1 make-check.gnu-cuda-openmpi.txt
make install 2>&1 |tee make-install.gnu-cuda-openmpi.txt
gnu version with with openMPI
Make sure these packages are installed locally: blas-3.2.1-4.el6.x86_64.rpm blas-devel-3.2.1-4.el6.x86_64.rpm lapack-3.2.1-4.el6.x86_64.rpm lapack-devel-3.2.1-4.el6.x86_64.rpm
cd /sw/BigDFT/1.6.0-intel/src/bigdft-1.6.0-gnu-openmpi
module load mpi/openMPI/1.4.3-gnu
module load NetCDF/4.1.3-gnu
module load ATLAS/3.9.39
module load cuda/4.0
module load opencl/cudasdk-4
module load opencl/opencl-1.1
./configure --prefix=/sw/BigDFT/1.6.0-gnu-openmpi CXX=mpic++ CC=mpicc FC=mpif90 F77=mpif77 --with-netcdf-path=/sw/netCDF/4.1.3-gnu-openmpi 2>&1 |tee configure.gnu-openmpi.txt
(Not needed: --with-mpi-include=-I/sw/openMPI/1.4.3-gnu/include)
>>>>>>>>>>>>>>>>
Basics:
C compiler: mpicc
C flags:
CXX compiler: mpic++
CXX flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64:-L/sw/ATLAS/3.9.39/lib
Linked libraries:
NVidia Cuda Compiler: /usr/local/cuda/bin//nvcc
NVCC flags: -g -O3 -DHAVE_CONFIG_H -I. -I.. -arch sm_13
NVCC C Compiler:
Cuda emulation: no
Trace functions: no
Verbose: no
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-openmpi
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
BigDFT configure summary:
========================
Basics:
Fortran90 compiler: mpif90
Fortran90 compiler name:
Fortran90 flags: -g -O2
Fortran77 compiler: mpif77
Fortran77 flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64:-L/sw/ATLAS/3.9.39/lib -L$(top_builddir)/libxc-1.1.0/src/.libs -L$(top_builddir)/libABINIT/src
Linked libraries: $(top_builddir)/src/libbigdft.a -labinit $(top_builddir)/src/libdft_common.a -lxc -llapack -lblas -larchive -lrt
Build:
Library ABINIT: yes
Library BigDFT: yes
Main binaries (cluster...):yes
Minima hopping binary: no
atom and pseudo binaries: no
User documentation: yes
Devel. documentation: yes / no
Options:
Debug version: no
With MPI: yes
| include dir.:
| linker flags:
| linked libraries:
| MPI_INIT_THREAD avail: yes
| MPI2 support: yes
With optimised conv.: yes
With Cuda GPU conv.: no
| NVidia Cuda Compiler:
| Cuda flags:
With OpenCL support: no
With dgemmsy support: no
With libXC: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libxc-1.1.0/src
With libABINIT: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libABINIT/src
With libS_GPU: no
| internal built: no
| include dir.:
With ETSF_IO: no
| include dir.:
With libarchive: yes
| C flags:
| linked libraries: -larchive
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-openmpi
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
Fortran modules: ${prefix}/include/
Documentation: ${datarootdir}/doc/${PACKAGE_TARNAME}
>>>>>>>>>>>>>>>>>>>>>>>>
make 2>&1 |tee make.gnu-openmpi.txt
make check 2>&1 make-check.gnu-openmpi.txt
make install 2>&1 |tee make-install.gnu-openmpi.txt
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
gcc version
cd /sw/BigDFT/1.6.0-intel/src/bigdft-1.6.0-gnu
module load NetCDF/4.1.3-gnu
module load ATLAS/3.9.39
module load cuda/4.0
module load opencl/cudasdk-4
module load opencl/opencl-1.1
./configure --prefix=/sw/BigDFT/1.6.0-gnu CXX=c++ CC=gcc FC=gfortran F77=gfortran --with-netcdf-path=/sw/netCDF/4.1.3-gnu 2>&1 --disable-mpi |tee configure.gnu.txt
>>>>>>>>>>>>>>>>>>>>>>>>>>>
Basics:
C compiler: gcc
C flags:
CXX compiler: c++
CXX flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64:-L/sw/ATLAS/3.9.39/lib
Linked libraries:
NVidia Cuda Compiler: /usr/local/cuda/bin//nvcc
NVCC flags: -g -O3 -DHAVE_CONFIG_H -I. -I.. -arch sm_13
NVCC C Compiler:
Cuda emulation: no
Trace functions: no
Verbose: no
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
BigDFT configure summary:
========================
Basics:
Fortran90 compiler: gfortran
Fortran90 compiler name:
Fortran90 flags: -g -O2
Fortran77 compiler: gfortran
Fortran77 flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64:-L/sw/ATLAS/3.9.39/lib -L$(top_builddir)/libxc-1.1.0/src/.libs -L$(top_builddir)/libABINIT/src
Linked libraries: $(top_builddir)/src/libbigdft.a -labinit $(top_builddir)/src/libdft_common.a -lxc -llapack -lblas -larchive -lrt
Build:
Library ABINIT: yes
Library BigDFT: yes
Main binaries (cluster...):yes
Minima hopping binary: no
atom and pseudo binaries: no
User documentation: yes
Devel. documentation: yes / no
Options:
Debug version: no
With MPI: no
| include dir.: -I$(top_builddir)/src/modules
| linker flags:
| linked libraries:
| MPI_INIT_THREAD avail:
| MPI2 support:
With optimised conv.: yes
With Cuda GPU conv.: no
| NVidia Cuda Compiler:
| Cuda flags:
With OpenCL support: no
With dgemmsy support: no
With libXC: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libxc-1.1.0/src
With libABINIT: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libABINIT/src
With libS_GPU: no
| internal built: no
| include dir.:
With ETSF_IO: no
| include dir.:
With libarchive: yes
| C flags:
| linked libraries: -larchive
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
Fortran modules: ${prefix}/include/
Documentation: ${datarootdir}/doc/${PACKAGE_TARNAME}
>>>>>>>>>>>>>>>>>>>>>>>>>>>
make 2>&1 |tee make.gnu.txt
make check 2>&1 make-check.gnu.txt
make install 2>&1 |tee make-install.gnu.txt
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Intel versions - incomplete
Intel version with intel MPI and Cuda support - Install not complete....
./configure --help
cd /sw/BigDFT/1.6.0-intel/src/bigdft-1.6.0-intel-cuda-intelmpi
module load intel-cmkl-11/11.1.072
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load intel-cc-11/11.1.072
module load intel-fc-11/11.1.072
module load cuda/4.0
module load opencl/cudasdk-4
module load opencl/opencl-1.1
module load NetCDF/4.1.3-gnu
module load ATLAS/3.9.39
./configure --enable-cuda-gpu --with-cuda-path=/usr/local/cuda --prefix=/sw/BigDFT/1.6.0-intel-cuda-intelmpi CXX=mpiicpc CC=mpiicc FC=mpiifort F77=mpif77 --with-netcdf-path=/sw/netCDF/4.1.3-gnu --with-mpi-include=-I/sw/sdev/intel/mpi/4.0.0.027/x86_64/include64 2>&1 |tee configure.intel-cuda-intelmpi.txt
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Basics:
C compiler: mpiicc
C flags:
CXX compiler: mpiicpc
CXX flags: -g -O2
Linker flags: -L/sw/ATLAS/3.9.39/lib:-L/usr/local/cuda/lib64:-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t
Linked libraries:
NVidia Cuda Compiler:
NVCC flags: -g -O3 -DHAVE_CONFIG_H -I. -I.. -arch sm_13
NVCC C Compiler:
Cuda emulation: no
Trace functions: no
Verbose: no
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-intel-cuda-intelmpi
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
BigDFT configure summary:
========================
Basics:
Fortran90 compiler: mpiifort
Fortran90 compiler name:
Fortran90 flags: -g -assume noold_unit_star
Fortran77 compiler: mpif77
Fortran77 flags: -O2
Linker flags: -L/sw/ATLAS/3.9.39/lib:-L/usr/local/cuda/lib64:-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -L/usr/local/cuda/lib64 -L$(top_builddir)/libxc-1.1.0/src/.libs -L$(top_builddir)/libABINIT/src
Linked libraries: $(top_builddir)/src/libbigdft.a -labinit $(top_builddir)/src/libdft_common.a -lxc $(top_builddir)/S_GPU/src/.libs/libs_gpu.a -lstdc++ -lcudart -lcublas -llapack -lblas -larchive -lrt
Build:
Library ABINIT: yes
Library BigDFT: yes
Main binaries (cluster...):yes
Minima hopping binary: no
atom and pseudo binaries: no
User documentation: yes
Devel. documentation: yes / no
Options:
Debug version: no
With MPI: yes
| include dir.: -I/sw/sdev/intel/mpi/4.0.0.027/x86_64/include64
| linker flags:
| linked libraries:
| MPI_INIT_THREAD avail: yes
| MPI2 support: yes
With optimised conv.: yes
With Cuda GPU conv.: yes
| NVidia Cuda Compiler: /usr/local/cuda/bin/nvcc
| Cuda flags:
With OpenCL support: no
With dgemmsy support: no
With libXC: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libxc-1.1.0/src
With libABINIT: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libABINIT/src
With libS_GPU: yes
| internal built: yes
| include dir.: -DCUBLAS_USE_THUNKING -I$(top_srcdir)/S_GPU/src
With ETSF_IO: no
| include dir.:
With libarchive: yes
| C flags:
| linked libraries: -larchive
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-intel-cuda-intelmpi
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
Fortran modules: ${prefix}/include/
Documentation: ${datarootdir}/doc/${PACKAGE_TARNAME}
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
make 2>&1 |tee make.intel-cuda-intelmpi.txt
This gives error...check it out..
make check 2>&1 make-check.intel-cuda-intelmpi.txt
make install 2>&1 |tee make-install.intel-cuda-intelmpi.txt
Install not complete....
Intel version with intel mpi - incomplete
Intel version with cuda - incomplete
Intel version - incomplete
Other attempts - incomplete
./configure --enable-cuda-gpu --with-cuda-path=/usr/local/cuda 2>&1 |tee configure.log ./configure --with-ext-linalg " -lmkl_intel64 -lmkl_lapack" --with-ext-linalg-path="-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/64" --prefix=/sw/BigDFT/1.6.0-intel FC=mpif90 --with-blas --with-blas-path "/sw/sdev/intel/Compiler/11.1/072/mkl/lib/64" --enable-cuda-gpu --with-cuda-path=/usr/local/cuda 2>&1 |tee configure.log --enable-cuda-gpu / --enable-opencl --prefix=DIR ../configure --with-ext-linalg "-lmkl_em64t -lmkl_lapack" --with-ext-linalg-path="-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t" --prefix=/sw/BigDFT/1.6.0-intel FC=mpif90 --enable-cuda-gpu --enable-opencl --enable-cuda-gpu --enable-opencl --with-cuda-path --with-cuda-cflags --with-ocl-path --prefix option is provided also to specify the destination directory for installation
gnu version with openMPI and Cuda support
This caused a lot of issues. However, thanks to Todd Chruchwood from SGI who managed to configure it by commenting out lines 8272-8277 of configure. #if test $withlinalg = "no" && test $withblas = "no" ; then # as_fn_error "\"No blas library. Use --with-blas-path or check your Blas installation.\"" "$LINENO" 5 #fi #if test $withlinalg = "no" && test $withlapack = "no" ; then # as_fn_error "\"No lapack library. Use --with-lapack-path or check your Lapack installation. Linalg: $LINALG_LIBS\"" "$LINENO" 5 #fi
cd /sw/BigDFT/1.6.0-intel/src/bigdft-1.6.0-gnu
module load cuda/4.0
module load opencl/cudasdk-4
module load opencl/opencl-1.1
module load mpi/openMPI/1.4.3-gnu
module load NetCDF/4.1.3-gnu
module load ATLAS/3.9.39
./configure --enable-cuda-gpu --with-cuda-path=/usr/local/cuda --prefix=/sw/BigDFT/1.6.0-gnu-openmpi-cuda CXX=mpic++ CC=mpicc FC=mpif90 F77=mpif77 --with-netcdf-path=/sw/netCDF/4.1.3-gnu 2>&1 |tee configure.gnu-openmpi-cuda.txt
(Not neded: --with-mpi-include=-I/sw/openMPI/1.4.3-gnu/include)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Basics:
C compiler: mpicc
C flags:
CXX compiler: mpic++
CXX flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64
Linked libraries:
NVidia Cuda Compiler: /usr/local/cuda/bin//nvcc
NVCC flags: -g -O3 -DHAVE_CONFIG_H -I. -I.. -arch sm_13
NVCC C Compiler:
Cuda emulation: no
Trace functions: no
Verbose: no
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-openmpi-cuda
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
BigDFT configure summary:
========================
Basics:
Fortran90 compiler: mpif90
Fortran90 compiler name:
Fortran90 flags: -g -O2
Fortran77 compiler: mpif77
Fortran77 flags: -g -O2
Linker flags: -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib -L$(top_builddir)/libxc-1.1.0/src/.libs -L$(top_builddir)/libABINIT/src
Linked libraries: $(top_builddir)/src/libbigdft.a -labinit $(top_builddir)/src/libdft_common.a -lxc $(top_builddir)/S_GPU/src/.libs/libs_gpu.a -lstdc++ -lcudart -lcublas -lrt
Build:
Library ABINIT: yes
Library BigDFT: yes
Main binaries (cluster...):yes
Minima hopping binary: no
atom and pseudo binaries: no
User documentation: yes
Devel. documentation: yes / no
Options:
Debug version: no
With MPI: yes
| include dir.:
| linker flags:
| linked libraries:
| MPI_INIT_THREAD avail: yes
| MPI2 support: yes
With optimised conv.: yes
With Cuda GPU conv.: yes
| NVidia Cuda Compiler: /usr/local/cuda/bin/nvcc
| Cuda flags:
With OpenCL support: no
With dgemmsy support: no
With libXC: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libxc-1.1.0/src
With libABINIT: yes
| internal built: yes
| include dir.: -I$(top_builddir)/libABINIT/src
With libS_GPU: yes
| internal built: yes
| include dir.: -DCUBLAS_USE_THUNKING -I$(top_srcdir)/S_GPU/src
With ETSF_IO: no
| include dir.:
With libarchive: warn
| C flags:
| linked libraries:
Installation paths:
Source code location: .
Prefix: /sw/BigDFT/1.6.0-gnu-openmpi-cuda
Exec prefix: ${prefix}
Binaries: ${exec_prefix}/bin
Static libraries: ${exec_prefix}/lib
Fortran modules: ${prefix}/include/
Documentation: ${datarootdir}/doc/${PACKAGE_TARNAME}
Warnings:
libarchive: disabled by tests
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
make 2>&1 | tee make-gnu-openmpi-cuda.log.txt