Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Current »

VASP is a plane wave ab initio code for quantum mechanical molecular dynamics. It is highly scalable and shows very good parallel performance for a variety of chemical and materials science calculations. VASP is available to gowonda users who already have a VASP license. VASP 5.2 requires a separate license and currently there is only 1 user (Dr. Yun Wang) licensed to use this on gowonda.

http://cms.mpi.univie.ac.at/vasp/ http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html

VASP requires that the BLAS package is installed on the computer.

module load vasp/vasp541-gnu-openmpi

To use vasp on the cluster, one needs to be in the unix group named "vasp". Please contact the HPC admin to request this.

VASP-gnu

Not installed currently

VASP-gnu Install Directory

cd /sw/VASP/AIBN/5.2/vasp.5.lib-gnu

VASP-mpi-intel

Usage:

 module load vasp/5.2vasp-mpi-intel

Sample pbs script

#!/bin/bash -l
#PBS -m abe
#PBS -V
### Mail to user
#PBS -M emailAddress@griffith.edu.au
### Job name
#PBS -N test_vasp
#PBS -W group_list=vasp
#PBS -l walltime=00:30:00
### Number of nodes:Number of CPUs:Number of threads per node
#PBS -l select=2:ncpus=2:mpiprocs=2:mem=8g
# This job's working directory
echo Working directory is $PBS_O_WORKDIR
cd $PBS_O_WORKDIR
source $HOME/.bashrc
module load vasp/5.2vasp-mpi-intel
module list
echo "Starting job"
echo Running on host `hostname`
echo Time is `date`
echo Directory is `pwd`
#echo This jobs runs on the following processors:
echo `cat $PBS_NODEFILE`
##mpirun  -machinefile   $PBS_NODEFILE  /export/home/snumber/pbs/mpi/p1.out
/sw/sdev/intel/mpi/4.0.0.027/x86_64/intel64/bin/mpirun -r ssh -f  $PBS_NODEFILE  vasp
echo "Done with job"

Install instructions

Instructions
============

  • change the Makefile for vasp and libs to use ifort and mpiifort
  • make sure the right links to mkl libraries are put in the appropriate place
  • load the following modules
    module load intel-mpi/4.0.0.027
    module load intel-tools-11/11.1.072
    module load fftw/3.2.2-intel
  • Just do make first in the lib and then in the vasp directory.

vasp.5.lib-mpi-intel

cd /sw/VASP/5.2.12/vasp.5.lib-mpi-intel
cp makefile.linux_ifc_P4 Makefile

module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

 module list
Currently Loaded Modulefiles:
  1) intel-mpi/4.0.0.027       2) intel-cc-11/11.1.072      3) intel-fc-11/11.1.072      4) intel-cmkl-11/11.1.072    5) intel-tools-11/11.1.072   6) fftw/3.2.2-intel


vi Makefile

Change the following to....
>>>>>>>>>>>>>
# C-preprocessor
CPP     = mpiicc -E -P -C $*.F >$*.f
FC=mpiifort
>>>>>>>>>>>>>>

make 2>&1 |tee make_vasp_lib_mpi_intel.txt

vasp.5.2mpi-intel

module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

cd /sw/VASP/5.2.12/vasp.5.2mpi-intel
cp makefile.linux_ifc_P4 Makefile



comment out the following in Makefile:

Item #

Pre Change

Post Change

1

-

FC=mpiifort

2

-

 BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread

3

-

LAPACK= -lmkl_lapack

4

-

FFT3D   = fft3dfurth.o fft3dlib.o

5

-

 FC=mpiifort 
 FCL=$(FC)

6

-

 CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \ 
   -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \ 
   -DMPI_BLOCK=8000

7

-

LIB     = -L../vasp.5.lib-mpi-intel -ldmy  \
      ../vasp.5.lib-mpi-intel/linpack_double.o $(LAPACK) \
      $(SCA) $(BLAS)

8

-

FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o 
 diff Makefile makefile.linux_ifc_P4
53c53
< FC=mpiifort
---
> FC=ifc
126,128d125
< #####BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl -lpthread
< ##BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl -lpthread
< BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
133c130
< ##BLAS=  /opt/libs/libgoto/libgoto.so
---
> BLAS=  /opt/libs/libgoto/libgoto.so
136c133
< ##LAPACK= ../vasp.5.lib/lapack_double.o
---
> LAPACK= ../vasp.5.lib/lapack_double.o
139c136
< LAPACK= -lmkl_lapack
---
> #LAPACK= -lmkl_lapack
143,144c140,141
< LIB  = -L../vasp.5.lib-intel -ldmy \
<      ../vasp.5.lib-intel/linpack_double.o $(LAPACK) \
---
> LIB  = -L../vasp.5.lib -ldmy \
>      ../vasp.5.lib/linpack_double.o $(LAPACK) \
175,177c172,173
< ##FC=mpif77
< FC=mpiifort
< FCL=$(FC)
---
> #FC=mpif77
> #FCL=$(FC)
194,197c190,193
< CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
<      -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
<      -DMPI_BLOCK=8000
< #    -DRPROMU_DGEMV  -DRACCMU_DGEMV
---
> #CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
> #     -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc -DNGZhalf \
> #     -DMPI_BLOCK=8000
> ##    -DRPROMU_DGEMV  -DRACCMU_DGEMV
216,218c212,214
< LIB     = -L../vasp.5.lib-mpi-intel -ldmy  \
<       ../vasp.5.lib-mpi-intel/linpack_double.o $(LAPACK) \
<       $(SCA) $(BLAS)
---
> #LIB     = -L../vasp.5.lib -ldmy  \
> #      ../vasp.5.lib/linpack_double.o $(LAPACK) \
> #      $(SCA) $(BLAS)
221c217
< FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o
---
> #FFT3D   = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o


make 2>&1 |tee make_vasp52_mpi_intel.txt

vasp-serial

vasp.5.lib-intel

cd /sw/VASP/5.2.12
tar -zxvf /data1/vasp.5.lib.tar.gz
mv vasp.5.lib vasp.5.lib-intel
cd /sw/VASP/5.2.12/vasp.5.lib-intel
cp makefile.linux_ifc_P4 Makefile

module purge
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel

vi Makefile

Change the following ....
>>>>>>>>>>>>>
 diff Makefile makefile.linux_ifc_P4
18,19c18,19
< CPP     = icc -E -P -C $*.F >$*.f
< FC=ifort
---
> CPP     = gcc -E -P -C $*.F >$*.f
> FC=ifc
51c51
<       icc $(CFLAGS) -c $*.c
---
>       $(CC) $(CFLAGS) -c $*.c
>>>>>>>>>>>>>>

make 2>&1 |tee make_vasp_lib_intel.txt

vasp.5.2-intel

module purge
module load intel-cc-11/11.1.072
module load fftw/3.2.2-intel
module load  intel-cmkl-11/11.1.072


module load intel-tools-11/11.1.072
module load  fftw/3.3-alpha-intel
module unload  intel-mpi/4.0.0.027
 module list
Currently Loaded Modulefiles:
  1) intel-cc-11/11.1.072      2) intel-fc-11/11.1.072      3) intel-cmkl-11/11.1.072    4) intel-tools-11/11.1.072   5) fftw/3.3-alpha-intel

cd /sw/VASP/5.2.12
tar -zxvf /data1/vasp.5.2.12.tar.gz
mv vasp.5.2 vasp.5.2-intel
cd /sw/VASP/5.2.12/vasp.5.2-intel
cp makefile.linux_ifc_P4 Makefile

>>>>>>>
 diff Makefile makefile.linux_ifc_P4
53c53
< FC=ifort
---
> FC=ifc
126d125
< BLAS=-L/sw/sdev/intel/Compiler/11.1/072/mkl/lib/em64t -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
131c130
< #BLAS=  /opt/libs/libgoto/libgoto.so
---
> BLAS=  /opt/libs/libgoto/libgoto.so
134c133
< LAPACK= ../vasp.5.lib-intel/lapack_double.o
---
> LAPACK= ../vasp.5.lib/lapack_double.o
141,142c140,141
< LIB  = -L../vasp.5.lib-intel -ldmy \
<      ../vasp.5.lib-intel/linpack_double.o $(LAPACK) \
---
> LIB  = -L../vasp.5.lib -ldmy \
>      ../vasp.5.lib/linpack_double.o $(LAPACK) \

>>>>>>>>>

make 2>&1 |tee make_vasp52_serial_intel.txt


VASP-5.4.1


module purge
module load intel-mpi/4.0.0.027
module load intel-tools-11/11.1.072
module load fftw/3.2.2-intel
cd /sw/VASP/5.4.1/intel-serial
cp arch/makefile.include.linux_intel_serial makefile.include
vi makefile.include

Add this: MKLROOT=/sw/sdev/intel/Compiler/11.1/072/mkl
make

This will build the standard, gamma-only, and non-collinear version
of VASP one after the other.

Patch
=====
module load  vasp/5.4.1vasp-intel-serial
cd /sw/VASP/5.4.1/intel-serial
gunzip the patch
e.g gunzip patch.5.4.1.03082016.gz
patch -p0 < patch.5.4.1.14032016
patch -p0 <patch.5.4.1.03082016 


Special Patch (for Tim):
Replace the files in src by the updated ones from patch_vasp.5.4.1.05Feb16.tgz (including the newly created file tim_parameters.F) and replace/modify also the "hidden" file .objects so as to contain the item tim_parameters.o right after subdft3.o.


VASP 5.4.4

Note: Awoonga is a different HPC managed by QRIS. Awoonga has been replaced by a newer HPC called Tinaroo. Information as of 18/01/23; Tinaroo is soon to be replaced by a newer HPC called Bunya. Bunya will have VASP 6. VASP 5.4.4 will not be available on Bunya. View FAQ Qs38 for details on Awoonga: FAQ - Griffith HPC Cluster#GriffithHPCCluster-Qs38%3AHowdoIgetstartedontheawoongacluster

awoonga node only
module load mkl/2017.3
module load intel/2017.4
module load lapack/3.6.0
cp arch/makefile.include.linux_intel_serial makefile.include
vi makefile.include

Add this: 
MKLROOT=/opt/intel/composer_xe_2017.4/mkl
make 2>&1 | tee makeLog.txt

Sample PBS script

#!/bin/bash
#PBS -N vaspstd
#PBS -m abe
#PBS -M YourEmail@griffithuni.edu.au
#PBS -l select=1:ncpus=1:mem=18gb,walltime=96:00:00
module load vasp/vasp541-gnu-openmpi
alias vasp="vasp_std"
cd $PBS_O_WORKDIR
vasp_std;


Another Sample PBS Script

#!/bin/bash
#PBS -m abe
#PBS -M EMAIL@griffith.edu.au
#PBS -N WestPap
#PBS -q workq
#PBS -l select=1:ncpus=1:mem=5gb,walltime=4:00:00
cd  $PBS_O_WORKDIR
module load vasp/vasp541-gnu-openmpi
echo "Starting job"
vasp_std < /dev/null > vasp.debug
echo "Done with Job"
  • No labels