R
Usage
module load R/3.4.0-gu (awoonga) module load R/3.4.0 (gowonda) Old Packages: ============= module load R/3.3.3 OR module load R/3.2.3 OR module load R/3.2.0 module load R/3.4.0 There are other older versions as well. module avail R
Accessing R on the gowonda cluster
Install Dir
/sw/R/2.13.0
Module load usage:
module load R/2.13.0
Installation notes
cd /sw/R tar -zxvf /data1/R-2.13.0.tar.gz cd /sw/R/R-2.13.0 ./configure --prefix=/sw/R/2.13.0 --with-blas="-L/sw/ATLAS/3.9.39/lib -lptf77blas -lpthread -latlas" --enable-R-shlib --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/usr/java/default 2>&1 | tee log.configure_R.txt >>>>>>>>>> R is now configured for x86_64-unknown-linux-gnu Source directory: . Installation directory: /sw/R/2.13.0 C compiler: gcc -std=gnu99 -g -O2 Fortran 77 compiler: gfortran -g -O2 C++ compiler: g++ -g -O2 Fortran 90/95 compiler: gfortran -g -O2 Obj-C compiler: Interfaces supported: X11 External libraries: readline, BLAS(ATLAS) Additional capabilities: PNG, JPEG, NLS Options enabled: shared R library, shared BLAS, R profiling, Java Recommended packages: yes >>>>>>>>>>>> make 2>&1 | tee log.make_R make check or make check-devel or make check-all 2>&1 | tee log.make_CheckALL_R.txt make install 2>&1 | tee log.make_install_R.txt cp -rp library lib include doc m4 modules po tests tools ../2.13.0
R install.packages('akima') * DONE (akima)
session info
> sessionInfo() R version 2.13.0 (2011-04-13) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base > R.Version() $platform [1] "x86_64-unknown-linux-gnu" $arch [1] "x86_64" $os [1] "linux-gnu" $system [1] "x86_64, linux-gnu" $status [1] "" $major [1] "2" $minor [1] "13.0" $year [1] "2011" $month [1] "04" $day [1] "13" $`svn rev` [1] "55427" $language [1] "R" $version.string [1] "R version 2.13.0 (2011-04-13)"
R Compilation Details
mkdir /usr/local/src/R wget http://cran.csiro.au/src/base/R-2/R-2.13.0.tar.gz tar -xf R-2.13.0.tar.gz cd /usr/local/src/R/R-2.13.0 make distclean ./configure --prefix=/usr/local/src/apps/R-2.13.0 --with-blas="-L/sw/ATLAS/3.9.39/lib -lptf77blas -lpthread -latlas" --enable-R-shlib --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/usr/java/default 2>&1 | tee log.configure_R [If no shared / dynamic libraries are needed, simply use: ./configure --prefix=/usr/local/src/apps/R-2.13.0] make 2>&1 | tee log.make_R make check or make check-devel or make check-all 2>&1 | tee log.make_CheckALL_R make install 2>&1 | tee log.make_install_R
This configure command above goves the following summary: R is now configured for x86_64-unknown-linux-gnu Source directory: . Installation directory: /usr/local/src/apps/R-2.13.0 C compiler: gcc -std=gnu99 -g -O2 Fortran 77 compiler: gfortran -g -O2 C++ compiler: g++ -g -O2 Fortran 90/95 compiler: gfortran -g -O2 Obj-C compiler: Interfaces supported: X11 External libraries: readline, BLAS(ATLAS) Additional capabilities: PNG, JPEG, NLS Options enabled: shared R library, shared BLAS, R profiling, Java Recommended packages: yes configure: WARNING: you cannot build DVI versions of the R manuals configure: WARNING: you cannot build DVI versions of all the help pages configure: WARNING: you cannot build info or HTML versions of the R manuals configure: WARNING: you cannot build PDF versions of the R manuals configure: WARNING: you cannot build PDF versions of all the help pages
Other Stuff to consider
atlas library
module load ATLAS/3.9.39 --with-blas="-L/sw/ATLAS/3.9.39/lib -lf77blas -latlas" For systems with multiple processors it is possible to use a multi-threaded version of ATLAS, by specifying --with-blas="-L/sw/ATLAS/3.9.39/lib -lptf77blas -lpthread -latlas"
mkl library
module load intel-cmkl-11/11.2.137 module load intel-cc-11/11.1.072 R can be linked to a sequential version of MKL by MKL_LIB_PATH=/sw/sdev/intel/ComposerXE/composerxe-2011.2.137/mkl/lib/intel64/ export LD_LIBRARY_PATH=$MKL_LIB_PATH MKL="-L${MKL_LIB_PATH} -lmkl_gf_lp64 -lmkl_sequential -lmkl_lapack -lmkl_core" ./configure --with-blas="$MKL" --with-lapack Threaded MKL may be used by replacing the line defining the variable MKL with MKL="-L${MKL_LIB_PATH} -lmkl_gf_lp64 -lmkl_gnu_thread \ -lmkl_lapack -lmkl_core -liomp5 -lpthread" The default number of threads will be chosen by the OpenMP* software, but can be controlled by setting OMP_NUM_THREADS or MKL_NUM_THREADS. Static MKL may be used with MKL=" -L${MKL_LIB_PATH} \ -Wl,--start-group \ ${MKL_LIB_PATH}/libmkl_gf_lp64.a \ ${MKL_LIB_PATH}/libmkl_gnu_thread.a \ ${MKL_LIB_PATH}/libmkl_core.a \ -Wl,--end-group \ -lgomp -lpthread" The default linking model, which is also used by version 9 of MKL, can be used by --with-blas="-lmkl -lguide -lpthread" This is multi-threaded, but in version 9 the number of threads defaults to 1. It can be increased by setting OMP_NUM_THREADS. (Thanks to Andy Liaw for the information.)
Configuration options
configure has many options: running ./configure --help ‘--with-x’ use the X Window System [yes] ‘--x-includes=DIR’ X include files are in DIR ‘--x-libraries=DIR’ X library files are in DIR ‘--with-readline’ use readline library (if available) [yes] ‘--enable-R-profiling’ attempt to compile support for Rprof() [yes] ‘--enable-memory-profiling’ attempt to compile support for Rprofmem() and tracemem() [no] ‘--enable-R-shlib’ build R as a shared/dynamic library [no] ‘--enable-BLAS-shlib’ build the BLAS as a shared/dynamic library [yes, except on AIX] You can use ‘--without-foo’ or ‘--disable-foo’ for the negatives. You will want to use ‘--disable-R-profiling’ if you are building a profiled executable of R (e.g. with ‘-pg)’. Flag ‘--enable-R-shlib’ causes the make process to build R as a dynamic (shared) library, typically called ‘libR.so’, and link the main R executable ‘R.bin’ against that library. This can only be done if all the code (including system libraries) can be compiled into a dynamic library, and there may be a performance1 penalty. If you need to re-configure R with different options you may need to run make clean or even make distclean before doing so.
config.site
The default optimization settings chosen for CFLAGS etc are conservative. It is likely that using ‘-mtune’ will result in significant performance improvements on recent CPUs (especially for ‘ix86’): one possibility is to add ‘-mtune=native’ for the best possible performance on the machine on which R is being installed: if the compilation is for a site-wide installation, it may still be desirable to use something like ‘-mtume=core2’. It is also possible to increase the optimization levels to ‘-O3’: however for many versions of the compilers this has caused problems in at least one CRAN package. For platforms with both 64- and 32-bit support, it is likely that LDFLAGS="-L/usr/local/lib64 -L/usr/local/lib" is appropriate since most (but not all) software installs its 64-bit libraries in ‘/usr/local/lib64’. To build a 32-bit version of R on ‘x86_64’ with Fedora 14 we used CC="gcc -m32" CXX="g++ -m32" F77="gfortran -m32" FC=${F77} OBJC=${CC} LDFLAGS="-L/usr/local/lib" LIBnn=lib 64-bit versions of Linux are built with support for files > 2Gb, and 32-bit versions will be if possible unless ‘--disable-largefile’ is specified.
Install R packages
Automatically download the package
Installing the akima package: R >install.packages('akima') akima is now installed in R's library subfolder, with all its dependencies intact. You may load it using library(akima) OR use: install.packages('akima', dependencies=TRUE, repos='http://cran.rstudio.com/') install.packages(c("EIAdata", "gdata", "ggmap", "ggplot2"))
Manually download the package
install.packages("akima", type="source") e.g: install.packages("/tmp/Rtmp9Jcjte/downloaded_packages/akima_0.5-4.tar.gz", repos=NULL) >>>>>>>>>>>>>> [root@n027 tmp]# R R version 2.13.0 (2011-04-13) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > install.packages("/tmp/akima_0.5-4.tar.gz", repos=NULL) * installing *source* package âakimaâ ... ** libs gfortran -fpic -g -O2 -c akima.new.f -o akima.new.o gfortran -fpic -g -O2 -c akima433.f -o akima433.o gfortran -fpic -g -O2 -c akima697.f -o akima697.o gfortran -fpic -g -O2 -c idbvip.f -o idbvip.o gfortran -fpic -g -O2 -c idcldp.f -o idcldp.o gfortran -fpic -g -O2 -c idgrid.f -o idgrid.o gfortran -fpic -g -O2 -c idlctn.f -o idlctn.o gfortran -fpic -g -O2 -c idpdrv.f -o idpdrv.o gfortran -fpic -g -O2 -c idptip.f -o idptip.o gfortran -fpic -g -O2 -c idptli.f -o idptli.o gfortran -fpic -g -O2 -c idsfft.f -o idsfft.o gfortran -fpic -g -O2 -c idtang.f -o idtang.o gfortran -fpic -g -O2 -c idxchg.f -o idxchg.o gcc -std=gnu99 -I/usr/local/src/apps/R-2.13.0/lib64/R/include -I/usr/local/include -fpic -g -O2 -c init.c -o init.o gfortran -fpic -g -O2 -c tripack.f -o tripack.o gfortran -fpic -g -O2 -c ttidbs.f -o ttidbs.o gcc -std=gnu99 -shared -L/usr/local/lib64 -o akima.so akima.new.o akima433.o akima697.o idbvip.o idcldp.o idgrid.o idlctn.o idpdrv.o idptip.o idptli.o idsfft.o idtang.o idxchg.o init.o tripack.o ttidbs.o -lgfortran -lm -L/usr/local/src/apps/R-2.13.0/lib64/R/lib -lR installing to /usr/local/src/apps/R-2.13.0/lib64/R/library/akima/libs ** R ** data ** preparing package for lazy loading ** help *** installing help indices ** building package indices ... ** testing if installed package can be loaded * DONE (akima) >>>>>>>
Post Install
vi /sw/R/2.13.0/lib64/R/bin/R R_HOME_DIR=/sw/R/2.13.0/lib64/R #R_HOME_DIR=/usr/local/src/apps/R-2.13.0/lib64/R
Additional packages
You may need to add this in ~/.R/Makevars to compile some of the packages successfully.
CXX14 = g++ -std=c++1y -Wno-unused-variable -Wno-unused-function -fPIC
mvtnorm
> install.packages("mvtnorm", repos="http://R-Forge.R-project.org") trying URL 'http://R-Forge.R-project.org/src/contrib/mvtnorm_0.9-9992.tar.gz' Content type 'application/x-gzip' length 315853 bytes (308 Kb) opened URL ================================================== downloaded 308 Kb * installing *source* package âmvtnormâ ... ** libs gcc -std=gnu99 -I/sw/R/2.13.0/lib64/R/include -I/usr/local/include -fpic -g -O2 -c miwa.c -o miwa.o gfortran -fpic -g -O2 -c mvt.f -o mvt.o gcc -std=gnu99 -I/sw/R/2.13.0/lib64/R/include -I/usr/local/include -fpic -g -O2 -c randomF77.c -o randomF77.o gfortran -fpic -g -O2 -c tvpack.f -o tvpack.o gcc -std=gnu99 -I/sw/R/2.13.0/lib64/R/include -I/usr/local/include -fpic -g -O2 -c tvpackAux.c -o tvpackAux.o gcc -std=gnu99 -shared -L/usr/local/lib64 -o mvtnorm.so miwa.o mvt.o randomF77.o tvpack.o tvpackAux.o -lgfortran -lm -L/sw/R/2.13.0/lib64/R/lib -lR installing to /sw/R/2.13.0/lib64/R/library/mvtnorm/libs ** R ** inst ** preparing package for lazy loading ** help *** installing help indices ** building package indices ... ** testing if installed package can be loaded * DONE (mvtnorm) The downloaded packages are in â/tmp/RtmpVkCqG5/downloaded_packagesâ Updating HTML index of packages in '.Library' Making packages.html ... done
corpcor
R-3.2.3 Installation
module load library/atlas/3.10.2 module load jdk/1.8.0_66 ./configure --prefix=/sw/R/R-3.2.3 --with-blas="-L/sw/library/atlas/3.10.2/my_build_dir/lib -lptf77blas -lpthread -latlas" --with-lapack --enable-R-shlib --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/sw/sdev/jdk/jdk1.8.0_66 2>&1 | tee log.configure_R make 2>&1 | tee log.make_R make check 2>&1 | tee log.makeCheck_R make install 2>&1 | tee log.make_install_R.txt cp -rp library lib include doc m4 modules po tests tools cp -rp library lib include doc m4 modules po tests tools ../R-3.2.3/ >>>>>>>>> module display R/3.2.3 ------------------------------------------------------------------- /sw/com/modulefiles/R/3.2.3: module-whatis Set up environment for R version 3.2.3 conflict R/3.0.2 conflict R/3.2.0 conflict R/3.2.3old module load jdk/1.8.0_66 module load library/gdal/1.9.2 module load library/proj/4.8.0 module load library/geos/3.4.2 module load jags/4.0.1 module load library/atlas/3.10.2 prepend-path PATH /sw/R/R-3.2.3/bin prepend-path PATH /sw/R/R-3.2.3/lib64/R/bin prepend-path LD_LIBRARY_PATH /sw/R/R-3.2.3/lib64/R/lib prepend-path INCLUDE /sw/R/R-3.2.3/lib64/R/include prepend-path MANPATH /sw/R/R-3.2.3/share/man ------------------------------------------------------------------- >>>>>>>> module load R/3.2.3 wget https://cran.r-project.org/src/contrib/pbkrtest_0.4-6.tar.gz module load R/3.2.3 install.packages('lme4', dependencies=TRUE, repos='http://cran.rstudio.com/') install.packages("/tmp/pbkrtest_0.4-6.tar.gz", repos=NULL) install.packages('caret', dependencies=TRUE, repos='http://cran.rstudio.com/')
Rmpi
module load mpi/openMPI/1.8.5-gcc4.9.0 R CMD INSTALL --configure-args="--with-Rmpi-include=/sw/openMPI/1.8.5-gcc4.9.0/include --with-Rmpi-libpath=/sw/openMPI/1.8.5-gcc4.9.0/lib/openmpi --with-Rmpi-type=OPENMPI" /tmp/Rmpi_0.6-5.tar.gz
gpuR
module load cuda/7.5.18 install.packages('gpuR', dependencies=TRUE, repos='http://cran.rstudio.com/')
rgdal
R CMD INSTALL rgdal
feseR
install.packages("devtools") library(devtools) install_github("enriquea/feseR")
R-3.3.3 Installation
Pre-Requisite
Pre-Requisite ============ >>>>>>>>>>>Install xy to take care of lzma dependency >>>>>>>>>>>>>>>>>>>>>>>>>> xz-5.3.3 tar -zxvf xz-5.2.3.tar.gz ./configure --prefix=/sw/library/xz/5.3.3 make -j3 make install >>>>>>>>>>>>pcre>>>>>>>>>>>>>>> pcre-8.39 bunzip2 pcre-8.39.tar.bz2 tar -xvf pcre-8.39.tar ./configure --enable-utf8 --prefix=/sw/pcre/8.39 make -j3 make install >>>>>>>>curl>>> wget https://curl.haxx.se/download/curl-7.53.1.tar.gz ./configure --prefix=/sw/misc/curl/7.53.1 make -j3 make install >>>>>>bzip2>>>>>>>>> got error related to bizp2 “/sw/library/bzip2/1.0.6/lib/libbz2.a: could not read symbols: Bad value” >> >>>>>>bzip2 fix>>>>>>>>> cd /sw/library/bzip2/src/bzip2-1.0.6fpic vi Makefile CFLAGS=-Wall -Winline -O2 -g $(BIGFILES) -fPIC make 2>&1 | tee makeLog.txt make install PREFIX=/sw/library/bzip2/1.0.6fpic 2>&1 | tee makeInstallLog.txt >>>>>>>
Actual Installation
module purge module load library/zlib/1.2.8 module load library/bzip2/1.0.6fpic module load library/xz/5.3.3 module load pcre/8.39 module load misc/curl/7.53.1 module load ATLAS/3.9.39 module load jdk/1.8.0_66 ./configure --prefix=/sw/R/3.3.3 '--with-cairo' '--with-jpeglib' '--with-readline' '--with-tcltk' '--with-blas' '--with-lapack' '--enable-R-profiling' '--enable-R-shlib' '--enable-memory-profiling' --enable-R-shlib --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/sw/sdev/jdk/jdk1.8.0_66/jre 2>&1 | tee log.configure_R.txt make 2>&1 | tee makeLog.txt make install 2>&1 | tee makeInstallLog.txt
Ref: http://pj.freefaculty.org/blog/?p=315
R-3.4.0
module purge module load library/zlib/1.2.8 module load library/bzip2/1.0.6fpic module load library/xz/5.3.3 module load pcre/8.39 module load misc/curl/7.53.1 module load ATLAS/3.9.39 module load jdk/1.8.0_66 module load gcc/4.9.3 module load misc/openssl/1.0.2 ./configure --prefix=/sw/R/3.4.0 '--with-cairo' '--with-jpeglib' '--with-readline' '--with-tcltk' '--with-blas' '--with-lapack' '--enable-R-profiling' '--enable-R-shlib' '--enable-memory-profiling' --enable-R-shlib --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/sw/sdev/jdk/jdk1.8.0_66/jre LDFLAGS=" -L/sw/pcre/8.39/lib" CPPFLAGS="-I/sw/pcre/8.39/include" 2>&1 | tee log.configure_R.txt make 2>&1 | tee makeLog.txt make install 2>&1 | tee makeInstallLog.txt
Useful Commands to install additional packages in R
install.packages("foobarbaz") ap <- available.packages() setRepositories() install.packages("phyloseq") library(devtools) install_github("packageauthor/foobarbaz") install_bitbucket("packageauthor/foobarbaz") install_gitorious("packageauthor/foobarbaz") install.packages("Rbbg", repos = "http://r.findata.org") install.packages("lubridate", dependencies=TRUE, repos='http://cran.rstudio.com/') source("https://bioconductor.org/biocLite.R") biocLite("preprocessCore") install_url("http://cran.r-project.org/src/contrib/Archive/sentiment/sentiment_0.2.tar.gz") R CMD INSTALL /tmp/RcppParallel_4.3.20.tar.gz
Sample PBS jobs
Serial jobs
#!/bin/bash ###################################################################### #### This section defines options for the pbs batching system ###################################################################### #PBS -m abe #PBS -M YourEmail@griffith.edu.au #PBS -q workq #PBS -l select=1:ncpus=1:mem=2gb,walltime=60:00:00 #PBS -N vivaxIBMS3 # ###Load the pgi compilers. module load compilers/pgi-32bit-12.4 ###Load the R module load R/4.0.3 ##################################################################### #### This section is for my debugging purposes (not required) ###################################################################### echo Running on host `hostname` echo Time is `date` ###################################################################### #### This section is setting up and running your executable or script ###################################################################### cd $PBS_O_WORKDIR ###cd /export/home/s12345/pbs/R/ echo Directory is `pwd` source $HOME/.bashrc R CMD BATCH '--args a=1 b=c(2,5,6)' test.R test.out #CMD BATCH options which tells it to immediately run an R program #instead of presenting an interactive prompt #R.out is the screen output #results.Rout is the R program output
More example here: http://confluence.rcs.griffith.edu.au:8080/display/v20zCluster/R#R-RunningRjobsserially
R mpi jobs
Check: http://confluence.rcs.griffith.edu.au:8080/display/GHPC/Rmpi
R And cuda:
R gputools is not currently installed as it needs EMI Photonics' CULA libraries.
http://brainarray.mbni.med.umich.edu/Brainarray/rgpgpu/ http://cran.r-project.org/web/packages/gputools/index.html
If the gputools package is installed, it will provide R interfaces to a handful of common statistical algorithms. These algorithms are implemented in parallel using a mixture of Nvidia's CUDA langauge, Nvidia's CUBLAS library, and EMI Photonics' CULA libraries. On a computer equiped with an Nvidia GPU some of these functions may be substantially more efficient than native R routines.
Sample R script for test (test.R)
Source: https://www.r-bloggers.com/2007/08/including-arguments-in-r-cmd-batch-mode/
##First read in the arguments listed at the command line args=(commandArgs(TRUE)) ##args is now a list of character vectors ## First check to see if arguments are passed. ## Then cycle through each element of the list and evaluate the expressions. if(length(args)==0){ print("No arguments supplied.") ##supply default values a = 1 b = c(1,1,1) }else{ for(i in 1:length(args)){ eval(parse(text=args[[i]])) } } print(a*2) print(b*3)
Sample script to run on flashlite cluster
This sample will only work on flashlite (UQ) cluster. If you are using flashlite, you may modify this template
#! /bin/bash # #PBS -m e #PBS -N BigMemRjob #PBS -l walltime=40:00:00 #PBS -q BigMemory ###Request 1.3TB of memory #PBS -l nodes=1:ppn=24,mem=1331200mb,vmem=1331200mb #PBS -A qris-gu source $HOME/.bashrc module load R/3.2.3 cd /home/userNameonFlashLite/pbs export ROutfile=ROutfile001.txt ## RUN ## echo "=== start R-3.2.3 ===" Rscript --no-save myRscript.R &> $ROutfile echo "Completed\n"
List all R packages
length(.packages(all.available=TRUE)) ip <- as.data.frame(installed.packages()[, c(1, 3:4)]) rownames(ip) <- NULL ip <- ip[is.na(ip$Priority), 1:2, drop=FALSE] print(ip, row.names=FALSE) For example: > length(.packages(all.available=TRUE)) [1] 200 > ip <- as.data.frame(installed.packages()[, c(1, 3:4)]) > rownames(ip) <- NULL > ip <- ip[is.na(ip$Priority), 1:2, drop=FALSE] > print(ip, row.names=FALSE) Package Version abind 1.4-5 acepack 1.4.1 anchors 3.0-8 askpass 1.1
How to run R code in parallel : Parallelisation using dplyr and doParallel
Parallelisation using plyr and doParallel
We have plyr and DoParallel in R/4.0.3
Threads vs. cores
There is often a lot of confusion between CPU threads and cores. A CPU core is the actual computation unit. Threads are a way of multi-tasking, and allow multiple simultaneous tasks to share the same CPU core. Multiple threads do not substitute for multiple cores. Because of this, compute-intensive workloads (like R) are typically only focused on the number of CPU cores available, not threads. (Ref: https://jstaf.github.io/hpc-r/parallel/)
Example: module load R/4.0.3 > library(plyr) > library(doParallel) Loading required package: foreach Loading required package: iterators Loading required package: parallel >cores <- detectCores() >cores [1] 72 > registerDoParallel(cores=12) fake_func <- function(x) { Sys.sleep(0.1) return(x) } library(microbenchmark) microbenchmark( serial = llply(1:24, fake_func), parallel = llply(1:24, fake_func, .parallel = TRUE), times = 1 ) Unit: milliseconds expr min lq mean median uq max neval serial 2424.3580 2424.3580 2424.3580 2424.3580 2424.3580 2424.3580 1 parallel 226.2199 226.2199 226.2199 226.2199 226.2199 226.2199 1 >
Sample interactive pbs run
qsub -I -l select=1:ncpus=8:mem=12gb,walltime=1:11:00 -q small Once you are on the compute node: module load R/4.0.3 R library(plyr) library(doParallel) cores <- detectCores() cores [1] 12 registerDoParallel(cores=8) fake_func <- function(x) { Sys.sleep(0.1) return(x) } library(microbenchmark) microbenchmark( serial = llply(1:24, fake_func), parallel = llply(1:24, fake_func, .parallel = TRUE), times = 1 ) Unit: milliseconds expr min lq mean median uq max neval serial 2402.7702 2402.7702 2402.7702 2402.7702 2402.7702 2402.7702 1 parallel 320.0491 320.0491 320.0491 320.0491 320.0491 320.0491 1 >
Create R conda environment (Recommended way on the new cluster) to run R on the new cluster
Another easy way to install R and R packages is to use conda . Here are the steps
source /usr/local/bin/s3proxy.sh module load anaconda3/2024.06 conda create --name myRenv R=4.3.1 source activate myRenv To install packes like "tidyverse","lubridate","ggplot2", "tmap", "ggmap", "sf", "ggsci", "remotes", "raster", "readxl", "terra" Simply google with the package name and conda as search strings e.g. tidyverse conda You will see instructions to install these packages: conda install r::r-lubridate conda install r::r-tidyverse conda install conda-forge::r-ggplot2 conda install conda-forge::r-tmap conda install conda-forge::r-ggmap conda install conda-forge::r-sf conda install conda-forge::r-ggsci conda install conda-forge::r-remotes conda install conda-forge::r-raster conda install conda-forge::r-readxl conda install conda-forge::r-terra Or simply: conda create --name myRenv r=4.3.1 r-tidyverse r-ggplot2 r-tmap r-ggmap r-sf r-ggsci r-remotes r-raster r-readxl r-terra -c conda-forge Here is a sample pbs script: #!/bin/bash #PBS -m abe #PBS -M XXXX@griffithuni.edu.au #PBS -N MyTest #PBS -q workq #PBS -l select=1:ncpus=1:mem=4gb,walltime=01:00:00 module load anaconda3/2024.06 source activate myRenv module list cd $PBS_O_WORKDIR R '--save' <MAIN_CODEmodel1.R ##sleep 22 echo "Done with job" >>>>> R script can start with loading the packages like this: package_names <- c("tidyverse","lubridate","ggplot2", "tmap", "ggmap", "sf", "ggsci", "remotes", "raster", "readxl", "terra") # Loop to load the packages for (package_name in package_names) { if (!requireNamespace(package_name, quietly = TRUE)) { message(paste("Installing and loading", package_name)) install.packages(package_name, dependencies = TRUE) } library(package_name, character.only = TRUE) } <snip>
Reference
1. http://yusung.blogspot.com.au/2009/01/install-jags-and-rjags-in-fedora.html