Versions Compared

Old Version 3

changes.mady.by.user Indy Siva

Saved on

compared with

New Version Current

changes.mady.by.user Indy Siva

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

VASP requires that the BLAS package is installed on the computer.

VASP-gnu

Not installed currently

VASP-gnu Install Directory

No Format
cdmodule load /sw/VASP/AIBN/5.2/vasp.5.lib-gnu

VASP-mpi-intel

Usage:

No Format
 module load vasp/5.2vasp-mpi-intel

Sample pbs script

No Format
#!/bin/bash -l
#PBS -m abe
#PBS -V
### Mail to user
#PBS -M emailAddress@griffith.edu.au
### Job name
#PBS -N test_vasp
#PBS -W group_list=vasp
#PBS -l walltime=00:30:00
### Number of nodes:Number of CPUs:Number of threads per node
#PBS -l select=2:ncpus=2:mpiprocs=2:mem=8g
# This job's working directory
echo Working directory is $PBS_O_WORKDIR
cd $PBS_O_WORKDIR
source $HOME/.bashrc
module load vasp/5.2vasp-mpi-intel
module list
echo "Starting job"
echo Running on host `hostname`
echo Time is `date`
echo Directory is `pwd`
#echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
##mpirun  -machinefile   $PBS_NODEFILE  /export/home/snumber/pbs/mpi/p1.out
/sw/sdev/intel/mpi/4.0.0.027/x86_64/intel64/bin/mpirun -r ssh -f  $PBS_NODEFILE  vasp
echo "Done with job"

Install instructions

Instructions
============

  • change the Makefile for vasp and libs to use ifort and mpiifort
  • make sure the right links to mkl libraries are put in the appropriate place
  • load the following modules
    module load intel-mpi/4.0.0.027
    module load intel-tools-11/11.1.072
    module load fftw/3.2.2-intel
  • Just do make first in the lib and then in the vasp directory.

vasp.5.lib-mpi-intel

No Format
cd /sw/VASP/5.2.12/vasp.5.lib-mpi-intel
cp makefile.linux_ifc_P4 Makefile

module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

 module list
Currently Loaded Modulefiles:
  1) intel-mpi/4.0.0.027       2) intel-cc-11/11.1.072      3) intel-fc-11/11.1.072      4) intel-cmkl-11/11.1.072    5) intel-tools-11/11.1.072   6) fftw/3.2.2-intel


vi Makefile

Change the following to....
>>>>>>>>>>>>>
# C-preprocessor
CPP     = mpiicc -E -P -C $*.F >$*.f
FC=mpiifort
>>>>>>>>>>>>>>

make 2>&1 |tee make_vasp_lib_mpi_intel.txt

vasp.5.2mpi-intel

No Format
module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

cd /sw/VASP/5.2.12/vasp.5.2mpi-intel
cp makefile.linux_ifc_P4 Makefile



comment out the following in Makefile:

Item #

Pre Change

Post Change

1

-

FC=mpiifort

2

-

No Format
 BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread

3

-

No Format
LAPACK= -lmkl_lapack

4

-

No Format
FFT3D   = fft3dfurth.o fft3dlib.o

5

-

No Format
 FC=mpiifort
No Format
 FCL=$(FC)

6

-

No Format
 CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
No Format
   -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
No Format
   -DMPI_BLOCK=8000

7

-

No Format
LIB     = -L../vasp.5.lib-mpi-intel -ldmy  \
      ../vasp.5.lib-mpi-intel/linpack_double.o $(LAPACK) \
      $(SCA) $(BLAS)

8

-

No Format
FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
No Format
 diff Makefile makefile.linux_ifc_P4
53c53
< FC=mpiifort
---
> FC=ifc
126,128d125
< #####BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl -lpthread
< ##BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl -lpthread
< BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
133c130
< ##BLAS=  /opt/libs/libgoto/libgoto.so
---
> BLAS=  /opt/libs/libgoto/libgoto.so
136c133
< ##LAPACK= ../vasp.5.lib/lapack_double.o
---
> LAPACK= ../vasp.5.lib/lapack_double.o
139c136
< LAPACK= -lmkl_lapack
---
> #LAPACK= -lmkl_lapack
143,144c140,141
< LIBvasp/vasp541-gnu-openmpi

To use vasp on the cluster, one needs to be in the unix group named "vasp". Please contact the HPC admin to request this.

VASP-gnu

Not installed currently

VASP-gnu Install Directory

No Format
cd /sw/VASP/AIBN/5.2/vasp.5.lib-gnu

VASP-mpi-intel

Usage:

No Format
 module load vasp/5.2vasp-mpi-intel

Sample pbs script

No Format
#!/bin/bash -l
#PBS -m abe
#PBS -V
### Mail to user
#PBS -M emailAddress@griffith.edu.au
### Job name
#PBS -N test_vasp
#PBS -W group_list=vasp
#PBS -l walltime=00:30:00
### Number of nodes:Number of CPUs:Number of threads per node
#PBS -l select=2:ncpus=2:mpiprocs=2:mem=8g
# This job's working directory
echo Working directory is $PBS_O_WORKDIR
cd $PBS_O_WORKDIR
source $HOME/.bashrc
module load vasp/5.2vasp-mpi-intel
module list
echo "Starting job"
echo Running on host `hostname`
echo Time is `date`
echo Directory is `pwd`
#echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
##mpirun  -machinefile   $PBS_NODEFILE  /export/home/snumber/pbs/mpi/p1.out
/sw/sdev/intel/mpi/4.0.0.027/x86_64/intel64/bin/mpirun -r ssh -f  $PBS_NODEFILE  vasp
echo "Done with job"

Install instructions

Instructions
============

  • change the Makefile for vasp and libs to use ifort and mpiifort
  • make sure the right links to mkl libraries are put in the appropriate place
  • load the following modules
    module load intel-mpi/4.0.0.027
    module load intel-tools-11/11.1.072
    module load fftw/3.2.2-intel
  • Just do make first in the lib and then in the vasp directory.

vasp.5.lib-mpi-intel

No Format
cd /sw/VASP/5.2.12/vasp.5.lib-mpi-intel
cp makefile.linux_ifc_P4 Makefile

module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

 module list
Currently Loaded Modulefiles:
  1) intel-mpi/4.0.0.027       2) intel-cc-11/11.1.072      3) intel-fc-11/11.1.072      4) intel-cmkl-11/11.1.072    5) intel-tools-11/11.1.072   6) fftw/3.2.2-intel


vi Makefile

Change the following to....
>>>>>>>>>>>>>
# C-preprocessor
CPP     = mpiicc -E -P -C $*.F >$*.f
FC=mpiifort
>>>>>>>>>>>>>>

make 2>&1 |tee make_vasp_lib_mpi_intel.txt

vasp.5.2mpi-intel

No Format
module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

cd /sw/VASP/5.2.12/vasp.5.2mpi-intel
cp makefile.linux_ifc_P4 Makefile



comment out the following in Makefile:


Item #

Pre Change

Post Change

1

-

FC=mpiifort

2

-


No Format
 BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread


3

-


No Format
LAPACK= -lmkl_lapack


4

-


No Format
FFT3D   = fft3dfurth.o fft3dlib.o


5

-


No Format
 FC=mpiifort


No Format
 FCL=$(FC)


6

-


No Format
 CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \


No Format
   -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \


No Format
   -DMPI_BLOCK=8000


7

-


No Format
LIB     = -L../vasp.5.lib-mpi-intel -ldmy  \
      ../vasp.5.lib-mpi-intel/linpack_double.o $(LAPACK) \
      $(SCA) $(BLAS)


8

-


No Format
FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o



No Format
 diff Makefile makefile.linux_ifc_P4
53c53
< FC=mpiifort
---
> FC=ifc
126,128d125
< #####BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl -lpthread
< ##BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl -lpthread
< BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
133c130
< ##BLAS=  /opt/libs/libgoto/libgoto.so
---
> BLAS=  /opt/libs/libgoto/libgoto.so
136c133
< ##LAPACK= ../vasp.5.lib/lapack_double.o
---
> LAPACK= ../vasp.5.lib/lapack_double.o
139c136
< LAPACK= -lmkl_lapack
---
> #LAPACK= -lmkl_lapack
143,144c140,141
< LIB  = -L../vasp.5.lib-intel -ldmy \
<      ../vasp.5.lib-intel/linpack_double.o $(LAPACK) \
---
> LIB  = -L../vasp.5.lib -ldmy \
>      ../vasp.5.lib/linpack_double.o $(LAPACK) \
175,177c172,173
< ##FC=mpif77
< FC=mpiifort
< FCL=$(FC)
---
> #FC=mpif77
> #FCL=$(FC)
194,197c190,193
< CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
<      -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
<      -DMPI_BLOCK=8000
< #    -DRPROMU_DGEMV  -DRACCMU_DGEMV
---
> #CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
> #     -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
> #     -DMPI_BLOCK=8000
> ##    -DRPROMU_DGEMV  -DRACCMU_DGEMV
216,218c212,214
< LIB     = -L../vasp.5.lib-mpi-intel -ldmy  \
<       ../vasp.5.lib-mpi-intel/linpack_double.o $(LAPACK) \
<       $(SCA) $(BLAS)
---
> #LIB     = -L../vasp.5.lib-intel -ldmy  \
> <#      ../vasp.5.lib-intel/linpack_double.o $(LAPACK) \
---
> LIB
> #      $(SCA) $(BLAS)
221c217
< FFT3D   = -L../vasp.5.lib -ldmy \fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
---
> #FFT3D     ../vasp.5.lib/linpack_double.o $(LAPACK) \
175,177c172,173
< ##FC=mpif77
< FC=mpiifort
< FCL=$(FC)
---
> #FC=mpif77
> #FCL=$(FC)
194,197c190,193
< CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
<      -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
<      -DMPI_BLOCK=8000
< #    -DRPROMU_DGEMV  -DRACCMU_DGEMV
---
> #CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
> #     -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
> #     -DMPI_BLOCK=8000
> ##    -DRPROMU_DGEMV  -DRACCMU_DGEMV
216,218c212,214
< LIB     = -L../vasp.5.lib-mpi-intel -ldmy  \
<       ../vasp.5.lib-mpi-intel/linpack_double.o $(LAPACK) \
<       $(SCA) $(BLAS)
---
> #LIB     = -L../vasp.5.lib -ldmy  \
> #      ../vasp.5.lib/linpack_double.o $(LAPACK) \
> #      $(SCA) $(BLAS)
221c217
< FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
---
> #FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
No Format
make 2>&1 |tee make_vasp52_mpi_intel.txt

vasp-serial

vasp.5.lib-intel

No Format
= fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o


No Format
make 2>&1 |tee make_vasp52_mpi_intel.txt

vasp-serial

vasp.5.lib-intel

No Format
cd /sw/VASP/5.2.12
tar -zxvf /data1/vasp.5.lib.tar.gz
mv vasp.5.lib vasp.5.lib-intel
cd /sw/VASP/5.2.12/vasp.5.lib-intel
cp makefile.linux_ifc_P4 Makefile

module purge
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

vi Makefile

Change the following ....
>>>>>>>>>>>>>
 diff Makefile makefile.linux_ifc_P4
18,19c18,19
< CPP     = icc -E -P -C $*.F >$*.f
< FC=ifort
---
> CPP     = gcc -E -P -C $*.F >$*.f
> FC=ifc
51c51
<       icc $(CFLAGS) -c $*.c
---
>       $(CC) $(CFLAGS) -c $*.c
>>>>>>>>>>>>>>

make 2>&1 |tee make_vasp_lib_intel.txt

vasp.5.2-intel

No Format
module purge
module load intel-cc-11/11.1.072
module load fftw/3.2.2-intel
module load  intel-cmkl-11/11.1.072


module load intel-tools-11/11.1.072
module load  fftw/3.3-alpha-intel
module unload  intel-mpi/4.0.0.027
 module list
Currently Loaded Modulefiles:
  1) intel-cc-11/11.1.072      2) intel-fc-11/11.1.072      3) intel-cmkl-11/11.1.072    4) intel-tools-11/11.1.072   5) fftw/3.3-alpha-intel

cd /sw/VASP/5.2.12
tar -zxvf /data1/vasp.5.2.lib12.tar.gz
mv vasp.5.lib2 vasp.5.lib2-intel
cd /sw/VASP/5.2.12/vasp.5.lib2-intel
cp makefile.linux_ifc_P4 Makefile

module purge
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

vi Makefile

Change the following ....
>>>>>>>>>>>>>
 diff Makefile makefile.linux_ifc_P4
18,19c18,19
< CPP     = icc -E -P -C $*.F >$*.f
< FC=ifort>>>>>>>
 diff Makefile makefile.linux_ifc_P4
53c53
< FC=ifort
---
> FC=ifc
126d125
< BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
131c130
< #BLAS=  /opt/libs/libgoto/libgoto.so
---
> BLAS=  /opt/libs/libgoto/libgoto.so
134c133
< LAPACK= ../vasp.5.lib-intel/lapack_double.o
---
> CPP    LAPACK= ../vasp.5.lib/lapack_double.o
141,142c140,141
< LIB  = gcc -E -P -C $*.F >$*.f
> FC=ifc
51c51
<       icc $(CFLAGS) -c $*.c
----L../vasp.5.lib-intel -ldmy \
<      ../vasp.5.lib-intel/linpack_double.o $(LAPACK) \
---
> LIB  = -L../vasp.5.lib -ldmy \
>      ../vasp.5.lib/linpack_double.o $(CCLAPACK) $(CFLAGS) -c $*.c
>>>>>>>>>>>>>>

\

>>>>>>>>>

make 2>&1 |tee make_vaspvasp52_libserial_intel.txt

...


VASP-5.

...

4.

...

1


No Format
module purge
module load intel-ccmpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel
module load  intel-cmkl-11/11.1.072


module load intel-tools-11/11.1.072
module load  fftw/3.3-alpha-intel
module unload  intel-mpi/4.0.0.027
 module list
Currently Loaded Modulefiles:
  1) intel-cc-11/11.1.072      2) intel-fc-11/11.1.072      3) intel-cmkl-11/11.1.072    4) intel-tools-11/11.1.072   5) fftw/3.3-alpha-intel

cd /sw/VASP/5.2.12
tar -zxvf /data1/vasp.5.2.12.tar.gz
mv vasp.5.2 vasp.5.2-intel
cd /sw/VASP/5.2.12/vasp.5.2-intel
cp makefile.linux_ifc_P4 Makefile

>>>>>>>
 diff Makefile makefile.linux_ifc_P4
53c53
< FC=ifort
---
> FC=ifc
126d125
< BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
131c130
< #BLAS=  /opt/libs/libgoto/libgoto.so
---
> BLAS=  /opt/libs/libgoto/libgoto.so
134c133
< LAPACK= ../vasp.5.lib-intel/lapack_double.o
---
> LAPACK= ../vasp.5.lib/lapack_double.o
141,142c140,141
< LIB  = -L../vasp.5.lib-intel -ldmy \
<      ../vasp.5.lib-intel/linpack_double.o $(LAPACK) \
---
> LIB  = -L../vasp.5.lib -ldmy \
>      ../vasp.5.lib/linpack_double.o $(LAPACK) \

>>>>>>>>>

make 2>&1 |tee make_vasp52_serial_intel.txt

 

VASP-5.4.1

 

No Format
module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel
cd /sw/VASP/5.4.1/intel-serial
cp arch/makefile.include.linux_intel_serial makefile.include
vi makefile.include

Add this: MKLROOT=/sw/sdev/intel/Compiler/11.1/072/mkl
make

This will build the standard, gamma-only, and non-collinear version
of VASP one after the other.
072
module load fftw/3.2.2-intel
cd /sw/VASP/5.4.1/intel-serial
cp arch/makefile.include.linux_intel_serial makefile.include
vi makefile.include

Add this: MKLROOT=/sw/sdev/intel/Compiler/11.1/072/mkl
make

This will build the standard, gamma-only, and non-collinear version
of VASP one after the other.

Patch
=====
module load  vasp/5.4.1vasp-intel-serial
cd /sw/VASP/5.4.1/intel-serial
gunzip the patch
e.g gunzip patch.5.4.1.03082016.gz
patch -p0 < patch.5.4.1.14032016
patch -p0 <patch.5.4.1.03082016 


Special Patch (for Tim):
Replace the files in src by the updated ones from patch_vasp.5.4.1.05Feb16.tgz (including the newly created file tim_parameters.F) and replace/modify also the "hidden" file .objects so as to contain the item tim_parameters.o right after subdft3.o.


VASP 5.4.4

Note: Awoonga is a different HPC managed by QRIS. Awoonga has been replaced by a newer HPC called Tinaroo. Information as of 18/01/23; Tinaroo is soon to be replaced by a newer HPC called Bunya. Bunya will have VASP 6. VASP 5.4.4 will not be available on Bunya. View FAQ Qs38 for details on Awoonga: FAQ - Griffith HPC Cluster#GriffithHPCCluster-Qs38%3AHowdoIgetstartedontheawoongacluster

No Format
awoonga node only
module load mkl/2017.3
module load intel/2017.4
module load lapack/3.6.0
cp arch/makefile.include.linux_intel_serial makefile.include
vi makefile.include

Add this: 
MKLROOT=/opt/intel/composer_xe_2017.4/mkl
make 2>&1 | tee makeLog.txt

Sample PBS script

No Format
#!/bin/bash
#PBS -q workq
#PBS -m bea
#PBS -M email@griffithuni.edu.au
#PBS -N VaspTest
#PBS -l select=1:ncpus=4:mpiprocs=4:mem=35gb,walltime=4:00:00
## load vasp module to setup env variables to run vasp
cd $PBS_O_WORKDIR
module load vasp/vasp541-gnu-openmpi
## get the number of allocated cpu cores
export nprocs=`cat $PBS_NODEFILE | wc -l`
## run vasp in parallel over $nprocs processors
###alias vasp="vasp_std"
mpiexec vasp_std < /dev/null > vasp.debug;
echo "Done with job"


Another Sample PBS Script

No Format
#!/bin/bash
#PBS -m abe
#PBS -M EMAIL@griffith.edu.au
#PBS -N WestPap
#PBS -q workq
#PBS -l select=1:ncpus=1:mem=5gb,walltime=4:00:00
cd  $PBS_O_WORKDIR
module load vasp/vasp541-gnu-openmpi
echo "Starting job"
vasp_std < /dev/null > vasp.debug
echo "Done with Job"