Initial Prep:
=============
 
Copy the images into the local container folder:
 
mkdir ~/Container
cp /sw/Containers/singularity/images/openfoam7-paraview56_latest.sif ~/Container/

Shell into the container and create launcher script:
====================================================
 
Initial setup
 
1. shell into the container
=============================
For example:

singularity  shell -B /scratch/s12345:/scratch --pwd /scratch/s12345:/scratch --pwd /scratch  /export/home/s12345/Container/openfoam7-paraview56_latest.sif
 
e.g:

singularity  shell -B /scratch/s5126720:/scratch --pwd /scratch/s5126720:/scratch --pwd /scratch  /export/home/s5126720/Container/openfoam7-paraview56_latest.sif
 
2. Create a variable file you can source (While shelled into the singularity container,, do this)

2a. Copy the bashrc file to scratch
cp   /opt/openfoam7/etc/bashrc /scratch

Modify the /scratch/bashrc file to suit your environment.

2b. (While Onceshelled in, thedo shellthis), create a variable file (openFOAMenv7.sh) with contents like this:


vi /scratch/openFOAMenvopenFOAMenv7.sh
export LD_LIBRARY_PATH=/opt/paraviewopenfoam56/lib:$LD_LIBRARY_PATH
export LIBRARY_PATH=/opt/paraviewopenfoam56/lib:$LIBRARY_PATH
export PATH=/opt/openfoam7:/opt/openfoam7/bin:/opt/paraviewopenfoam56/bin:$PATH
export   LDFLAGS=-L/opt/paraviewopenfoam56/lib:$LDFLAGS
export#export   PKG_CONFIG_PATH=/sw/library/loki/0.1.7/lib/pkgconfig:$PKG_CONFIG_PATH
export   INCLUDE=/opt/paraviewopenfoam56/include:$INCLUDE
export   CPLUS_INCLUDE_PATH=/opt/paraviewopenfoam56/include:$CPLUS_INCLUDE_PATH
export   CPATH=/opt/paraviewopenfoam56/include:$CPATH
export   C_INCLUDE_PATH=/opt/paraviewopenfoam56/include:$C_INCLUDE_PATH
export   FPATH=/opt/paraviewopenfoam56/include:$FPATH
export   CFLAGS=-I/opt/paraviewopenfoam56/include:$CFLAGS
export   CPPFLAGS=-I/opt/paraviewopenfoam56/include:$CPPFLAGS
export   CXXFLAGS=-I/opt/paraviewopenfoam56/include:$CXXFLAGS
export   FCFLAGS=-I/opt/paraviewopenfoam56/include:$FCFLAGS
export   FFLAGS=-I/opt/paraviewopenfoam56/include:$FFLAGS
source
/scratch/bashrc


3. Manual Test Run 
  ===================

 
(While shelled into the singularity container, do this)

source /scratch/openFOAMenvopenFOAMenv7.sh 
foamInfo -help

For example:
 
Singularity openfoam7-paraview56_latest.sif:/opt/openfoam7> foamInfo -help

Usage: foamInfo [OPTIONS] <name>
options:
  -all     | -a        list all tutorials that use <name> (otherwise maximum 10)

If the output looks correct, go to the next step

4. Create a Launcher Script;
===========================
Once you are happy that all step 1 ,2 and 3 works, you can create a launcherScript
While still inside the shell:
 
vi /scratch/openfoamLauncheropenfoamLauncher7.sh
 
An example of the content is this:

export LD_LIBRARY_PATH=/opt/paraviewopenfoam56/lib:$LD_LIBRARY_PATH
export LIBRARY_PATH=/opt/paraviewopenfoam56/lib:$LIBRARY_PATH
export PATH=/opt/openfoam7:/opt/openfoam7/bin:/opt/paraviewopenfoam56/bin:$PATH
export   LDFLAGS=-L/opt/paraviewopenfoam56/lib:$LDFLAGS
export#export   PKG_CONFIG_PATH=/sw/library/loki/0.1.7/lib/pkgconfig:$PKG_CONFIG_PATH
export   INCLUDE=/opt/paraviewopenfoam56/include:$INCLUDE
export   CPLUS_INCLUDE_PATH=/opt/paraviewopenfoam56/include:$CPLUS_INCLUDE_PATH
export   CPATH=/opt/paraviewopenfoam56/include:$CPATH
export   C_INCLUDE_PATH=/opt/paraviewopenfoam56/include:$C_INCLUDE_PATH
export   FPATH=/opt/paraviewopenfoam56/include:$FPATH
export   CFLAGS=-I/opt/paraviewopenfoam56/include:$CFLAGS
export   CPPFLAGS=-I/opt/paraviewopenfoam56/include:$CPPFLAGS
export   CXXFLAGS=-I/opt/paraviewopenfoam56/include:$CXXFLAGS
export   FCFLAGS=-I/opt/paraviewopenfoam56/include:$FCFLAGS
export   FFLAGS=-I/opt/paraviewopenfoam56/include:$FFLAGS
source#Put your run /scratch/bashrc
cd /scratch
#Put your run instructions/instructions/command below.
foamInfo -help

>>>>>>>>>>>>> 

make it executable:
chmod +x /scratch/openfoamLauncheropenfoamLauncher7.sh

Run it manually and check if it gives the expected reultsresults:

Check if it gives the expected results:
/scratch/openfoamLauncheropenfoamLauncher7.sh
(OR: try cd /scratch;./openfoamLauncher.sh)

5. Running it as a batch job:
==============================

Exit the shell and create this pbs script anywhere in space:
 
mkdir /scratch/s2940457/pbs
cd /scratch/s2940457/pbs
vi pbsopenfoam7.01pbs01

>>>>>>>>> >>>>>>>>> 

#!/bin/bash 
#PBS -m abe
#PBS -M YourEmail@griffith.edu.au
#PBS -N TestOpenFoamTestOpenFoam7
#PBS -q workq
#PBS -l select=1:ncpus=18:mem=2gb,walltime=3:00:00
cd  $PBS_O_WORKDIR
#An example is as below. Modify accordingly to suit your needs
singularity  exec -B /scratch/s5126720s2940457:/scratch --pwd /scratch/s5126720s2940457:/scratch --pwd /scratch  /export/home/s5126720s2940457/Container/openfoam7-paraview56_latest.sif "./openfoamLauncheropenfoamLauncher7.sh"
exit
sleep 2 
>>>>>> 
 
Submit the job and check the results (output and error files).
 
qsub pbsopenfoam7.01

pbs01  


If all goes well, then the only file that needs to be edited in the future is the openfoamLauncher7.sh

To run in parallel, you may look at online documentation.
>>>>>>>>>>>>>>
e.g
How to run in parallel:
https://www.youtube.com/watch?v=YdYHDMygNPU
>>>>>>>>>>