| Table of Contents |
|---|
Introduction
Usage
| No Format |
|---|
module load Delft3D/trunk2-ics2015 |
...
| No Format |
|---|
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Source from: svn co https://svn.oss.deltares.nl/repos/delft3d/trunk delft3trunk cd /sw/Delft3D/src unzip D3D_trunk.zip >>>>>>>>>>> chmod +x ./scripts_lgpl/linux/update_version.sh chmod +x ./scripts_lgpl/linux/make_revision.sh >>>>>>>>>>>> module load intel-ComposerXE/2015.2.164 module load intel-mpi/5.0.3.048 module load compilers/autoconf-2.69 module load NetCDF/4.3.3.1-intel_2015 export NETCDF_CFLAGS=/sw/NetCDF/4.3.3.1/ComposerXE2015/include export NETCDF_LIBS=/sw/NetCDF/4.3.3.1/ComposerXE2015/lib export I_MPI_F90=ifort export I_MPI_F77=ifort export UUIDLIBS_ADDITIONAL="/lib64/libuuid.so.1" export UUIDLIBS=/lib64/libuuid.so.1 cd /sw/Delft3D/src/trunk2/src sh autogen.sh -v 2>&1 | tee autoconfigLog.txt CFLAGS='-O2' CXXFLAGS='-O2' FFLAGS='-O2' FCFLAGS='-O2' ./configure --prefix=/sw/Delft3D/unknown 2>&1 | tee configureLog.txt >>>>>>>After the above step, add -luuid in the Makefile>>>>> vi Makefile FCLIBS = -L/sw/NetCDF/4.3.3.1/ComposerXE2015/lib:-L/sw...............snip ......-ldl -luuid >>>>>>>>> make all 2>&1 | tee makeLog.txt make install all 2>&1 | tee makeInstallLog.txt |
...
Sample pbs script
Sample script to run on flashlite cluster
| No Format |
|---|
###This is only for flashlite. It won't work with gowonda
#! /bin/bash
### Template PBS script to submit Delft3d job to queue gccm
#PBS -m e
#PBS -N 3d12wvnew
#PBS -l walltime=200:00:00
#PBS -l nodes=1:ppn=4:intel,mem=30gb,vmem=30gb
#PBS -A qris-gu
source $HOME/.bashrc
module load delft3d-rev5624
cd /gpfs1/groups/gccmss/project/delft3d/3D/12waves_3_new/
##input data file
export MDWFILE=fm_wave_bcw.mdw
export ARGFILE=config_flow2d3d.xml
## RUN ##
# start flow
echo "=== start Delft3D-FLOW in the background ==="
#$MPIRUNEXEC -np $NHOSTS d_hydro.exe $ARGFILE &
#deltares_hydro.exe $ARGFILE -keepXML &
#deltares_hydro.exe $ARGFILE &
d_hydro.exe $ARGFILE &
# start wave
echo "=== start Delft3D-WAVE in the background ==="
#$MPIRUNEXEC -np $NHOSTS wave.exe $MDWFILE 1
wave.exe $MDWFILE 1
echo "=== calculation finished ==="
echo "Finished job "`date`
|
Installation on the new Cluster : Version 62441
Initial Pre-Requisites
| No Format |
|---|
Environment: OS = RHEL 7.6 with intelParallelStudio2019update5 Initial Pre-requisites • Subversion client Available on the cluster • GNU Autotools >>>>> Already installed Development Tools rpm -qa|grep -i auto autofs-5.0.7-99.el7.x86_64 autoconf-2.69-11.el7.noarch automake-1.13.4-3.el7.noarch libsss_autofs-1.16.2-13.el7_6.8.x86_64 autogen-libopts-5.18-5.el7.x86_64 >>>> • GNU Libtool. rpm -qa|grep -i libtool libtool-2.4.2-22.el7_3.x86_64 libtool-ltdl-2.4.2-22.el7_3.x86_64 We also have libtool, libtool-ltdl and libtool-ltdl-devel • GNU C++ compiler (use Package Manager), version 3.4.6. Alternative: Intel C++ compiler, version 11.1 or higher. gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) • expat-devel rpm -qa|grep expat expat-2.1.0-10.el7_3.x86_64 expat-devel-2.1.0-10.el7_3.x86_64 • GNU Fortran compiler 4.9.1. version 4.8.5 20150623 • Mpich, version 3.1.2 or higher rpm -qa|grep mpich mpich-3.2-devel-3.2-2.el7.x86_64 mpich-3.2-3.2-2.el7.x86_64 • Lex & Yacc rpm -qa|egrep "flex|bison" flex-devel-2.5.37-6.el7.x86_64 bison-3.0.4-2.el7.x86_64 bison-devel-3.0.4-2.el7.x86_64 flex-2.5.37-6.el7.x86_64 • OpenSSL. rpm -qa|grep -i openssl pyOpenSSL-0.13.1-4.el7.x86_64 perl-Crypt-OpenSSL-RSA-0.28-7.el7.x86_64 openssl-libs-1.0.2k-16.el7_6.1.x86_64 openssl-devel-1.0.2k-16.el7_6.1.x86_64 perl-Crypt-OpenSSL-Bignum-0.04-18.el7.x86_64 perl-Crypt-OpenSSL-Random-0.04-21.el7.x86_64 openssl-1.0.2k-16.el7_6.1.x86_64 • "readline-devel" (RedHat/Fedora) rpm -qa|grep readline readline-devel-6.2-10.el7.x86_64 readline-6.2-10.el7.x86_64 • Ruby rpm -qa|grep -i ruby rubygem-rdoc-4.0.0-35.el7_6.noarch rubygem-io-console-0.4.2-35.el7_6.x86_64 rubygem-bigdecimal-1.2.0-35.el7_6.x86_64 rubygem-psych-2.0.0-35.el7_6.x86_64 ruby-irb-2.0.0.648-35.el7_6.noarch ruby-devel-2.0.0.648-35.el7_6.x86_64 rubygems-2.0.14.1-35.el7_6.noarch rubygem-json-1.7.7-35.el7_6.x86_64 ruby-libs-2.0.0.648-35.el7_6.x86_64 ruby-2.0.0.648-35.el7_6.x86_64 • NetCDF: netcdf-c-4.6.1 or above, netcdf-fortran-4.4.5 or above, with hdf5 enabled module load library/netcdf/netcdf-fortran/4.5.2 module load library/netcdf/netcdf-c/4.7.3 (module load library/netcdf/netcdf-cxx/4.3.1) (Compiled |
...
with gcc 4.8.5)
• uuid-dev package
rpm -qa|grep uuid
libuuid-devel-2.23.2-59.el7_6.1.x86_64
libuuid-2.23.2-59.el7_6.1.x86_64
|
Actual Install
| No Format |
|---|
module purge
module load intel/intelParallelStudio2019
module load library/xz/5.2.4
module load library/hdf5/5-1.10.5-intel
module load library/netcdf/netcdf-fortran/4.5.2-intel
module load library/netcdf/netcdf-cxx/4.3.1-intel
module load library/netcdf/netcdf-c/4.7.3-intel
module load mpi/mpich/3.3.2-intel
module list
Currently Loaded Modulefiles:
1) intel/intelParallelStudio2019 4) library/netcdf/netcdf-fortran/4.5.2-intel 7) mpi/mpich/3.3.2-intel
2) library/xz/5.2.4 5) library/netcdf/netcdf-cxx/4.3.1-intel
3) library/hdf5/5-1.10.5-intel 6) library/netcdf/netcdf-c/4.7.3-intel
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
sh autogen.sh 2>&1 | tee autogenLogs-intel.txt
./configure --prefix=/sw/Delft3D/62441intel 2>&1 | tee configureLogs-intel.txt
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/scripts_lgpl/linux
chmod +x *
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
#Perform FIX 1:
# You may skip FIX 2 as it is done and no longer needed:
#Perform FIX 3
make 2>&1 | tee makeLogs-intel.txt
make install 2>&1 | tee makeInstallLogs.txt
Some of the fixes:
==================
FIX 1:
=====
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
sh fixPTHread.sh
........Notes and Reference.........................
cat fixPTHread.sh
#Reference#https://lists.gnu.org/archive/html/emacs-devel/2019-05/msg00032.html
/bin/ld: //usr/lib64/libpthread.so.0: error adding symbols: DSO
missing from command line
collect2: error: ld returned 1 exit status
On Red Hat and Fedora I believe this is because of `-lrt`. I patch the
makefiles so '-lrt' becomes '-;rt -lpthread'. I do so (1) after
unpacking the tarball; and (2) after configure. It usually sidesteps
this error.
for file in $(find "$PWD" -iname 'Makefile')
do
echo "patching $file"
sed -e 's|-lrt|-lrt -lpthread|g' "$file" > "$file.fixed"
mv "$file.fixed" "$file"
touch -t 197001010000 "$file"
done
.................................
>>>>>>>>>>>>>>>>>>
FIX 2: Only needs to be done once.
======
mkdir /sw/intel/ps/2019up5/intelpython3/intel64/
ln -s /sw/intel/ps/2019up5/intelpython3/include/ /sw/intel/ps/2019up5/intelpython3/intel64/include
FIX 3: Manually compile two binaries. piece.f and concat.f
======
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/engines_gpl/waq/packages/waq_utils_f/src/interact
I had to do this manually:
/sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/libtool --tag=F77 --mode=compile mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include -c -o piece.lo piece.f
/sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/libtool --tag=F77 --mode=compile mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include -c -o concat.lo concat.f
mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include -frecursive -c concat.f -fPIC -o
>>>>>>>>>>>>>>>>>>>>>>>
|