...
Introduction
Usage
| No Format |
|---|
Notmodule working at the moment...Install issues. Please stay tuned. load Delft3D/trunk2-ics2015 New Cluster: module load delft3d/6244intel |
Installation
| No Format |
|---|
Had>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> toSource remove libtool from system and use the latest... yum remove libtool from: svn co https://svn.oss.deltares.nl/repos/delft3d/trunk delft3trunk module load library/libtool/2.4.2 module load mpi/mpich2/2.1.4.1p1-gnu cd /sw/Delft3D/delft3d_repository/src ./autogen.sh -v ./configure --prefix=`pwd` CC=icc CXX=icpc ldd libflow2d3d.so >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cd /sw/Delft3D/src unzip D3D_trunk.zip >>>>>>>>>>> chmod +x ./scripts_lgpl/linux/update_version.sh chmod +x ./scripts_lgpl/linux/make_revision.sh >>>>>>>>>>>> module load intel-ComposerXE/2015.2.164 module load intel-mpi/5.0.3.048 module load compilers/autoconf-2.69 module load NetCDF/4.23.13.1-intel_2015 export NETCDF_CFLAGS=/sw/NetCDF/4.23.13.1/ComposerXE2015/include export NETCDF_LIBS=/sw/NetCDF/4.23.13.1/ComposerXE2015/lib export I_MPI_F90=ifort export I_MPI_F77=ifort export UUIDLIBS_ADDITIONAL="/lib64/libuuid.so.1" export UUIDLIBS=/lib64/libuuid.so.1 cd /sw/Delft3D/src/trunk2/src sh autogen.sh -v 2>&1 | tee autoconfigLog.txt CFLAGS='-O2' CXXFLAGS='-O2' FFLAGS='-O2' FCFLAGS='-O2' ./configure --prefix=/sw/Delft3D/unknown 2>&1 | tee configureLog.txt >>>>>>>After the above step, add -luuid in the Makefile>>>>> vi Makefile FCLIBS = -L/sw/NetCDF/4.3.3.1/ComposerXE2015/lib:-L/sw...............snip ......-ldl -luuid >>>>>>>>> make all 2>&1 | tee makeLog.txt make install all 2>&1 | tee makeInstallLog.txt |
Sample pbs script
Sample script to run on flashlite cluster
| No Format |
|---|
###This is only for flashlite. It won't work with gowonda
#! /bin/bash
### Template PBS script to submit Delft3d job to queue gccm
#PBS -m e
#PBS -N 3d12wvnew
#PBS -l walltime=200:00:00
#PBS -l nodes=1:ppn=4:intel,mem=30gb,vmem=30gb
#PBS -A qris-gu
source $HOME/.bashrc
module load delft3d-rev5624
cd /gpfs1/groups/gccmss/project/delft3d/3D/12waves_3_new/
##input data file
export MDWFILE=fm_wave_bcw.mdw
export ARGFILE=config_flow2d3d.xml
## RUN ##
# start flow
echo "=== start Delft3D-FLOW in the background ==="
#$MPIRUNEXEC -np $NHOSTS d_hydro.exe $ARGFILE &
#deltares_hydro.exe $ARGFILE -keepXML &
#deltares_hydro.exe $ARGFILE &
d_hydro.exe $ARGFILE &
# start wave
echo "=== start Delft3D-WAVE in the background ==="
#$MPIRUNEXEC -np $NHOSTS wave.exe $MDWFILE 1
wave.exe $MDWFILE 1
echo "=== calculation finished ==="
echo "Finished job "`date`
|
Installation on the new Cluster : Version 62441
Initial Pre-Requisites
| No Format |
|---|
Environment: OS = RHEL 7.6 with intelParallelStudio2019update5
Initial Pre-requisites
• Subversion client
Available on the cluster
• GNU Autotools
>>>>>
Already installed Development Tools
rpm -qa|grep -i auto
autofs-5.0.7-99.el7.x86_64
autoconf-2.69-11.el7.noarch
automake-1.13.4-3.el7.noarch
libsss_autofs-1.16.2-13.el7_6.8.x86_64
autogen-libopts-5.18-5.el7.x86_64
>>>>
• GNU Libtool.
rpm -qa|grep -i libtool
libtool-2.4.2-22.el7_3.x86_64
libtool-ltdl-2.4.2-22.el7_3.x86_64
We also have libtool, libtool-ltdl and libtool-ltdl-devel
• GNU C++ compiler (use Package Manager), version 3.4.6.
Alternative: Intel C++ compiler, version 11.1 or higher.
gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)
• expat-devel
rpm -qa|grep expat
expat-2.1.0-10.el7_3.x86_64
expat-devel-2.1.0-10.el7_3.x86_64
• GNU Fortran compiler 4.9.1.
version 4.8.5 20150623
• Mpich, version 3.1.2 or higher
rpm -qa|grep mpich
mpich-3.2-devel-3.2-2.el7.x86_64
mpich-3.2-3.2-2.el7.x86_64
• Lex & Yacc
rpm -qa|egrep "flex|bison"
flex-devel-2.5.37-6.el7.x86_64
bison-3.0.4-2.el7.x86_64
bison-devel-3.0.4-2.el7.x86_64
flex-2.5.37-6.el7.x86_64
• OpenSSL.
rpm -qa|grep -i openssl
pyOpenSSL-0.13.1-4.el7.x86_64
perl-Crypt-OpenSSL-RSA-0.28-7.el7.x86_64
openssl-libs-1.0.2k-16.el7_6.1.x86_64
openssl-devel-1.0.2k-16.el7_6.1.x86_64
perl-Crypt-OpenSSL-Bignum-0.04-18.el7.x86_64
perl-Crypt-OpenSSL-Random-0.04-21.el7.x86_64
openssl-1.0.2k-16.el7_6.1.x86_64
• "readline-devel" (RedHat/Fedora)
rpm -qa|grep readline
readline-devel-6.2-10.el7.x86_64
readline-6.2-10.el7.x86_64
• Ruby
rpm -qa|grep -i ruby
rubygem-rdoc-4.0.0-35.el7_6.noarch
rubygem-io-console-0.4.2-35.el7_6.x86_64
rubygem-bigdecimal-1.2.0-35.el7_6.x86_64
rubygem-psych-2.0.0-35.el7_6.x86_64
ruby-irb-2.0.0.648-35.el7_6.noarch
ruby-devel-2.0.0.648-35.el7_6.x86_64
rubygems-2.0.14.1-35.el7_6.noarch
rubygem-json-1.7.7-35.el7_6.x86_64
ruby-libs-2.0.0.648-35.el7_6.x86_64
ruby-2.0.0.648-35.el7_6.x86_64
• NetCDF: netcdf-c-4.6.1 or above, netcdf-fortran-4.4.5 or above, with hdf5 enabled
module load library/netcdf/netcdf-fortran/4.5.2
module load library/netcdf/netcdf-c/4.7.3
(module load library/netcdf/netcdf-cxx/4.3.1)
(Compiled with gcc 4.8.5)
• uuid-dev package
rpm -qa|grep uuid
libuuid-devel-2.23.2-59.el7_6.1.x86_64
libuuid-2.23.2-59.el7_6.1.x86_64
|
Actual Install
| No Format |
|---|
module purge module load intel/intelParallelStudio2019 module load library/xz/5.2.4 module load library/hdf5/5-1.10.5-intel module load library/netcdf/netcdf-fortran/4.5.2-intel module load library/netcdf/netcdf-cxx/4.3.1-intel module load library/netcdf/netcdf-c/4.7.3-intel module load mpi/mpich/3.3.2-intel module list Currently Loaded Modulefiles: 1) intel/intelParallelStudio2019 4) library/netcdf/netcdf-fortran/4.5.2-intel 7) mpi/mpich/3.3.2-intel 2) library/xz/5.2.4 5) library/netcdf/netcdf-cxx/4.3.1-intel 3) library/hdf5/5-1.10.5-intel 6) library/netcdf/netcdf-c/4.7.3-intel cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel sh autogen.sh 2>&1 | tee autogenLogs-intel.txt ./configure --prefix=/sw/Delft3D/62441intel 2>&1 | tee configureLogs-intel.txt cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/scripts_lgpl/linux chmod +x * cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel #Perform FIX 1: # You may skip FIX 2 as it is done and no longer needed: #Perform FIX 3 make 2>&1 | tee makeLog.txt |
...
makeLogs-intel.txt
make install 2>&1 | tee makeInstallLogs.txt
Some of the fixes:
==================
FIX 1:
=====
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel
sh fixPTHread.sh
........Notes and Reference.........................
cat fixPTHread.sh
#Reference#https://lists.gnu.org/archive/html/emacs-devel/2019-05/msg00032.html
/bin/ld: //usr/lib64/libpthread.so.0: error adding symbols: DSO
missing from command line
collect2: error: ld returned 1 exit status
On Red Hat and Fedora I believe this is because of `-lrt`. I patch the
makefiles so '-lrt' becomes '-;rt -lpthread'. I do so (1) after
unpacking the tarball; and (2) after configure. It usually sidesteps
this error.
for file in $(find "$PWD" -iname 'Makefile')
do
echo "patching $file"
sed -e 's|-lrt|-lrt -lpthread|g' "$file" > "$file.fixed"
mv "$file.fixed" "$file"
touch -t 20912101610 "$file"
done
.................................
>>>>>>>>>>>>>>>>>>
FIX 2: Only needs to be done once.
======
mkdir /sw/intel/ps/2019up5/intelpython3/intel64/
ln -s /sw/intel/ps/2019up5/intelpython3/include/ /sw/intel/ps/2019up5/intelpython3/intel64/include
FIX 3: Manually compile two binaries. piece.f and concat.f
======
cd /sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/engines_gpl/waq/packages/waq_utils_f/src/interact
I had to do this manually:
/sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/libtool --tag=F77 --mode=compile mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include -c -o piece.lo piece.f
/sw/Delft3D/62441/src/Delft3D_newversion/Delft3D_Source_62441/src.intel/libtool --tag=F77 --mode=compile mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include -c -o concat.lo concat.f
mpif77 -DWITH_DELFTONLINE -I/sw/mpi/mpich/3.3.2-intel/include:-I/sw/library/netcdf/NetCDF-C/4.7.3-intel/include:-I/sw/library/netcdf/netcdf-cxx/4-4.3.1-intel/include:-I/sw/library/netcdf/NetCDF-Fortran/4.5.2-intel/include:-I/sw/library/hdf5/5-1.10.5-intel/include:-I/sw/library/xz/5.2.4/include -frecursive -c concat.f -fPIC -o
>>>>>>>>>>>>>>>>>>>>>>>
|