Delft3D

Owned by Indy Siva
Last updated: Dec 11, 2019
3 min read

Introduction


Usage

module load Delft3D/trunk2-ics2015
New Cluster: module load delft3d/6244intel

Installation

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Source from: svn co https://svn.oss.deltares.nl/repos/delft3d/trunk delft3trunk

cd /sw/Delft3D/src
unzip D3D_trunk.zip
>>>>>>>>>>>
chmod +x ./scripts_lgpl/linux/update_version.sh
chmod +x ./scripts_lgpl/linux/make_revision.sh
>>>>>>>>>>>>
module load intel-ComposerXE/2015.2.164
module load intel-mpi/5.0.3.048
module load compilers/autoconf-2.69
module load NetCDF/4.3.3.1-intel_2015
export NETCDF_CFLAGS=/sw/NetCDF/4.3.3.1/ComposerXE2015/include
export NETCDF_LIBS=/sw/NetCDF/4.3.3.1/ComposerXE2015/lib
export I_MPI_F90=ifort
export I_MPI_F77=ifort
export UUIDLIBS_ADDITIONAL="/lib64/libuuid.so.1"
export UUIDLIBS=/lib64/libuuid.so.1
cd /sw/Delft3D/src/trunk2/src
sh autogen.sh -v 2>&1 | tee autoconfigLog.txt
CFLAGS='-O2' CXXFLAGS='-O2' FFLAGS='-O2' FCFLAGS='-O2' ./configure --prefix=/sw/Delft3D/unknown 2>&1 | tee configureLog.txt

>>>>>>>After the above step, add -luuid in the Makefile>>>>>
 vi Makefile

FCLIBS =   -L/sw/NetCDF/4.3.3.1/ComposerXE2015/lib:-L/sw...............snip ......-ldl -luuid

>>>>>>>>>

 make all 2>&1 | tee makeLog.txt
make install all 2>&1 | tee makeInstallLog.txt 


Sample pbs script

Sample script to run on flashlite cluster

###This is only for flashlite. It won't work with gowonda
#! /bin/bash

### Template PBS script to submit Delft3d job to queue gccm
#PBS -m e
#PBS -N 3d12wvnew
#PBS  -l  walltime=200:00:00
#PBS  -l  nodes=1:ppn=4:intel,mem=30gb,vmem=30gb
#PBS  -A qris-gu

                              
source $HOME/.bashrc

module load delft3d-rev5624

cd /gpfs1/groups/gccmss/project/delft3d/3D/12waves_3_new/
##input data file
export MDWFILE=fm_wave_bcw.mdw
export ARGFILE=config_flow2d3d.xml


## RUN ##
# start flow
echo "=== start Delft3D-FLOW in the background ==="
#$MPIRUNEXEC -np $NHOSTS d_hydro.exe $ARGFILE &
#deltares_hydro.exe $ARGFILE -keepXML &
#deltares_hydro.exe $ARGFILE  &
d_hydro.exe $ARGFILE &

# start wave
echo "=== start Delft3D-WAVE in the background ==="
#$MPIRUNEXEC -np $NHOSTS wave.exe $MDWFILE 1 
wave.exe $MDWFILE 1
echo "=== calculation finished ==="
echo "Finished job "`date`


Installation on the new Cluster : Version 62441

Initial Pre-Requisites

Environment: OS = RHEL 7.6 with  intelParallelStudio2019update5

Initial Pre-requisites

•       Subversion client
Available on the cluster

•       GNU Autotools
>>>>>
Already installed  Development Tools
rpm -qa|grep -i auto
autofs-5.0.7-99.el7.x86_64
autoconf-2.69-11.el7.noarch
automake-1.13.4-3.el7.noarch
libsss_autofs-1.16.2-13.el7_6.8.x86_64
autogen-libopts-5.18-5.el7.x86_64
>>>>

•       GNU Libtool.
rpm -qa|grep -i libtool
libtool-2.4.2-22.el7_3.x86_64
libtool-ltdl-2.4.2-22.el7_3.x86_64
We also have libtool, libtool-ltdl and libtool-ltdl-devel

•       GNU C++ compiler (use Package Manager), version 3.4.6.
Alternative: Intel C++ compiler, version 11.1 or higher.
gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)

•       expat-devel
rpm -qa|grep expat
expat-2.1.0-10.el7_3.x86_64
expat-devel-2.1.0-10.el7_3.x86_64


•       GNU Fortran compiler 4.9.1.
version 4.8.5 20150623

•       Mpich, version 3.1.2 or higher
rpm -qa|grep mpich
mpich-3.2-devel-3.2-2.el7.x86_64
mpich-3.2-3.2-2.el7.x86_64


•       Lex & Yacc
rpm -qa|egrep "flex|bison"

flex-devel-2.5.37-6.el7.x86_64
bison-3.0.4-2.el7.x86_64
bison-devel-3.0.4-2.el7.x86_64
flex-2.5.37-6.el7.x86_64


•       OpenSSL.
rpm -qa|grep -i openssl
pyOpenSSL-0.13.1-4.el7.x86_64
perl-Crypt-OpenSSL-RSA-0.28-7.el7.x86_64
openssl-libs-1.0.2k-16.el7_6.1.x86_64
openssl-devel-1.0.2k-16.el7_6.1.x86_64
perl-Crypt-OpenSSL-Bignum-0.04-18.el7.x86_64
perl-Crypt-OpenSSL-Random-0.04-21.el7.x86_64
openssl-1.0.2k-16.el7_6.1.x86_64

•       "readline-devel" (RedHat/Fedora)
rpm -qa|grep readline
readline-devel-6.2-10.el7.x86_64
readline-6.2-10.el7.x86_64


•       Ruby
rpm -qa|grep -i ruby
rubygem-rdoc-4.0.0-35.el7_6.noarch
rubygem-io-console-0.4.2-35.el7_6.x86_64
rubygem-bigdecimal-1.2.0-35.el7_6.x86_64
rubygem-psych-2.0.0-35.el7_6.x86_64
ruby-irb-2.0.0.648-35.el7_6.noarch
ruby-devel-2.0.0.648-35.el7_6.x86_64
rubygems-2.0.14.1-35.el7_6.noarch
rubygem-json-1.7.7-35.el7_6.x86_64
ruby-libs-2.0.0.648-35.el7_6.x86_64
ruby-2.0.0.648-35.el7_6.x86_64


•       NetCDF: netcdf-c-4.6.1 or above, netcdf-fortran-4.4.5 or above, with hdf5 enabled
module load library/netcdf/netcdf-fortran/4.5.2
module load library/netcdf/netcdf-c/4.7.3
(module load library/netcdf/netcdf-cxx/4.3.1)

(Compiled with gcc 4.8.5)

•       uuid-dev package
rpm -qa|grep uuid
libuuid-devel-2.23.2-59.el7_6.1.x86_64
libuuid-2.23.2-59.el7_6.1.x86_64

Actual Install

module purge
module load intel/intelParallelStudio2019
module load library/xz/5.2.4
module load library/hdf5/5-1.10.5-intel
module load library/netcdf/netcdf-fortran/4.5.2-intel
module load library/netcdf/netcdf-cxx/4.3.1-intel
module load library/netcdf/netcdf-c/4.7.3-intel
module load  mpi/mpich/3.3.2-intel
module list
Currently Loaded Modulefiles:
  1) intel/intelParallelStudio2019               4) library/netcdf/netcdf-fortran/4.5.2-intel   7) mpi/mpich/3.3.2-intel
  2) library/xz/5.2.4                            5) library/netcdf/netcdf-cxx/4.3.1-intel
  3) library/hdf5/5-1.10.5-intel                 6) library/netcdf/netcdf-c/4.7.3-intel

cd  /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
sh autogen.sh 2>&1 | tee autogenLogs-intel.txt
./configure --prefix=/sw/Delft3D/62441intel  2>&1 | tee configureLogs-intel.txt
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/scripts_lgpl/linux
chmod +x *
cd  /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
#Perform FIX 1:
# You may skip FIX 2 as it is done and no longer needed:
#Perform FIX 3
make 2>&1 | tee makeLogs-intel.txt
make install 2>&1 | tee makeInstallLogs.txt

Some of the fixes:
==================

FIX 1:
=====
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
sh  fixPTHread.sh
........Notes and Reference.........................
cat fixPTHread.sh
#Reference#https://lists.gnu.org/archive/html/emacs-devel/2019-05/msg00032.html
/bin/ld: //usr/lib64/libpthread.so.0: error adding symbols: DSO
missing from command line
collect2: error: ld returned 1 exit status

On Red Hat and Fedora I believe this is because of `-lrt`. I patch the
makefiles so '-lrt' becomes '-;rt -lpthread'. I do so (1) after
unpacking the tarball; and (2) after configure. It usually sidesteps
this error.

for file in $(find "$PWD" -iname 'Makefile')
do
    echo "patching $file"
    sed -e 's|-lrt|-lrt -lpthread|g' "$file" > "$file.fixed"
    mv "$file.fixed" "$file"
    touch -t 20912101610 "$file"
done
.................................
>>>>>>>>>>>>>>>>>>
FIX 2: Only needs to be done once.
======
mkdir /sw/intel/ps/2019up5/intelpython3/intel64/
ln -s /sw/intel/ps/2019up5/intelpython3/include/ /sw/intel/ps/2019up5/intelpython3/intel64/include

FIX 3: Manually compile two binaries.  piece.f and concat.f
======

cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/engines_gpl/waq/packages/waq_utils_f/src/interact
I had to do this manually:

/sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/libtool --tag=F77   --mode=compile mpif77   -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include  -c -o piece.lo piece.f

/sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/libtool --tag=F77   --mode=compile mpif77   -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include  -c -o concat.lo concat.f

mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include  -frecursive -c concat.f  -fPIC -o

>>>>>>>>>>>>>>>>>>>>>>>


Reference

  1. https://oss.deltares.nl/web/delft3d
  2. https://oss.deltares.nl/web/delft3d/source-code