Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 21 Next »

Usage

module load R/3.3.3
OR
module load R/3.2.3
OR
module load R/3.2.0
module load R/3.4.0
There are other older versions as well.
module avail R

 

Griffith Uni R Users site

Griffith Uni R Users site: http://guru.forumotion.co.nz/

Accessing R on the gowonda cluster

Install Dir

/sw/R/2.13.0

Module load usage:

module load R/2.13.0

Installation notes

cd /sw/R
tar -zxvf /data1/R-2.13.0.tar.gz
cd /sw/R/R-2.13.0
 ./configure --prefix=/sw/R/2.13.0 --with-blas="-L/sw/ATLAS/3.9.39/lib -lptf77blas -lpthread -latlas" --enable-R-shlib  --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/usr/java/default   2>&1 | tee log.configure_R.txt

>>>>>>>>>>
R is now configured for x86_64-unknown-linux-gnu

  Source directory:          .
  Installation directory:    /sw/R/2.13.0

  C compiler:                gcc -std=gnu99  -g -O2
  Fortran 77 compiler:       gfortran  -g -O2

  C++ compiler:              g++  -g -O2
  Fortran 90/95 compiler:    gfortran -g -O2
  Obj-C compiler:

  Interfaces supported:      X11
  External libraries:        readline, BLAS(ATLAS)
  Additional capabilities:   PNG, JPEG, NLS
  Options enabled:           shared R library, shared BLAS, R profiling, Java

  Recommended packages:      yes
>>>>>>>>>>>>
make  2>&1 | tee log.make_R
make check
or
make check-devel
or
make check-all  2>&1 | tee log.make_CheckALL_R.txt
make  install 2>&1 | tee log.make_install_R.txt
cp -rp library lib include doc m4 modules po tests tools  ../2.13.0


R
install.packages('akima')

* DONE (akima)

session info

> sessionInfo()
R version 2.13.0 (2011-04-13)
Platform: x86_64-unknown-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base


> R.Version()
$platform
[1] "x86_64-unknown-linux-gnu"

$arch
[1] "x86_64"

$os
[1] "linux-gnu"

$system
[1] "x86_64, linux-gnu"

$status
[1] ""

$major
[1] "2"

$minor
[1] "13.0"

$year
[1] "2011"

$month
[1] "04"

$day
[1] "13"

$`svn rev`
[1] "55427"

$language
[1] "R"

$version.string
[1] "R version 2.13.0 (2011-04-13)"

R Compilation Details

mkdir /usr/local/src/R
wget http://cran.csiro.au/src/base/R-2/R-2.13.0.tar.gz
tar -xf R-2.13.0.tar.gz
cd /usr/local/src/R/R-2.13.0
make distclean
./configure --prefix=/usr/local/src/apps/R-2.13.0 --with-blas="-L/sw/ATLAS/3.9.39/lib -lptf77blas -lpthread -latlas" --enable-R-shlib  --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/usr/java/default   2>&1 | tee log.configure_R

[If no shared / dynamic libraries are needed, simply use: ./configure --prefix=/usr/local/src/apps/R-2.13.0]
make  2>&1 | tee log.make_R
make check
or
make check-devel
or
make check-all  2>&1 | tee log.make_CheckALL_R
make  install 2>&1 | tee log.make_install_R
This configure command above goves the following summary:

R is now configured for x86_64-unknown-linux-gnu

  Source directory:          .
  Installation directory:    /usr/local/src/apps/R-2.13.0

  C compiler:                gcc -std=gnu99  -g -O2
  Fortran 77 compiler:       gfortran  -g -O2

  C++ compiler:              g++  -g -O2
  Fortran 90/95 compiler:    gfortran -g -O2
  Obj-C compiler:

  Interfaces supported:      X11
  External libraries:        readline, BLAS(ATLAS)
  Additional capabilities:   PNG, JPEG, NLS
  Options enabled:           shared R library, shared BLAS, R profiling, Java

  Recommended packages:      yes

configure: WARNING: you cannot build DVI versions of the R manuals
configure: WARNING: you cannot build DVI versions of all the help pages
configure: WARNING: you cannot build info or HTML versions of the R manuals
configure: WARNING: you cannot build PDF versions of the R manuals
configure: WARNING: you cannot build PDF versions of all the help pages

Other Stuff to consider

atlas library

module load ATLAS/3.9.39

--with-blas="-L/sw/ATLAS/3.9.39/lib -lf77blas -latlas"
For systems with multiple processors it is possible to use a multi-threaded version of ATLAS,
by specifying
--with-blas="-L/sw/ATLAS/3.9.39/lib -lptf77blas -lpthread -latlas"

mkl library

module load intel-cmkl-11/11.2.137
module load intel-cc-11/11.1.072


R can be linked to a sequential version of MKL by
MKL_LIB_PATH=/sw/sdev/intel/ComposerXE/composerxe-2011.2.137/mkl/lib/intel64/
export LD_LIBRARY_PATH=$MKL_LIB_PATH
MKL="-L${MKL_LIB_PATH} -lmkl_gf_lp64 -lmkl_sequential -lmkl_lapack -lmkl_core"
./configure --with-blas="$MKL" --with-lapack


Threaded MKL may be used by replacing the line defining the variable MKL with
MKL="-L${MKL_LIB_PATH} -lmkl_gf_lp64 -lmkl_gnu_thread \
-lmkl_lapack -lmkl_core -liomp5 -lpthread"


The default number of threads will be chosen by the OpenMP* software, but can be controlled
by setting OMP_NUM_THREADS or MKL_NUM_THREADS.
Static MKL may be used with
MKL=" -L${MKL_LIB_PATH} \
-Wl,--start-group \
${MKL_LIB_PATH}/libmkl_gf_lp64.a \
${MKL_LIB_PATH}/libmkl_gnu_thread.a \
${MKL_LIB_PATH}/libmkl_core.a \
-Wl,--end-group \
-lgomp -lpthread"

The default linking model, which is also used by version 9 of MKL, can be used by
--with-blas="-lmkl -lguide -lpthread"
This is multi-threaded, but in version 9 the number of threads defaults to 1. It can be
increased by setting OMP_NUM_THREADS. (Thanks to Andy Liaw for the information.)

Configuration options

configure has many options: running
./configure --help

‘--with-x’
use the X Window System [yes]
‘--x-includes=DIR’
X include files are in DIR
‘--x-libraries=DIR’
X library files are in DIR
‘--with-readline’
use readline library (if available) [yes]
‘--enable-R-profiling’
attempt to compile support for Rprof() [yes]
‘--enable-memory-profiling’
attempt to compile support for Rprofmem() and tracemem() [no]
‘--enable-R-shlib’
build R as a shared/dynamic library [no]
‘--enable-BLAS-shlib’
build the BLAS as a shared/dynamic library [yes, except on AIX]
You can use ‘--without-foo’ or ‘--disable-foo’ for the negatives.
You will want to use ‘--disable-R-profiling’ if you are building a profiled executable
of R (e.g. with ‘-pg)’.
Flag ‘--enable-R-shlib’ causes the make process to build R as a dynamic (shared)
library, typically called ‘libR.so’, and link the main R executable ‘R.bin’ against that
library. This can only be done if all the code (including system libraries) can be compiled
into a dynamic library, and there may be a performance1 penalty.

If you need to re-configure R with different options you may need to run make clean or
even make distclean before doing so.

config.site

The default optimization settings chosen for CFLAGS etc are conservative. It is likely
that using ‘-mtune’ will result in significant performance improvements on recent CPUs
(especially for ‘ix86’): one possibility is to add ‘-mtune=native’ for the best possible
performance on the machine on which R is being installed: if the compilation is for a
site-wide installation, it may still be desirable to use something like ‘-mtume=core2’. It is
also possible to increase the optimization levels to ‘-O3’: however for many versions of the
compilers this has caused problems in at least one CRAN package.
For platforms with both 64- and 32-bit support, it is likely that
LDFLAGS="-L/usr/local/lib64 -L/usr/local/lib"
is appropriate since most (but not all) software installs its 64-bit libraries in
‘/usr/local/lib64’. To build a 32-bit version of R on ‘x86_64’ with Fedora 14 we used
CC="gcc -m32"
CXX="g++ -m32"
F77="gfortran -m32"
FC=${F77}
OBJC=${CC}
LDFLAGS="-L/usr/local/lib"
LIBnn=lib
64-bit versions of Linux are built with support for files > 2Gb, and 32-bit versions will
be if possible unless ‘--disable-largefile’ is specified.

Install R packages

Automatically download the package

Installing the akima package:

R
>install.packages('akima')
akima is now installed in R's library subfolder, with all its dependencies intact. You may load it using

library(akima)
OR use:
install.packages('akima', dependencies=TRUE, repos='http://cran.rstudio.com/')
 

Manually download the package

install.packages("akima", type="source")

e.g:

install.packages("/tmp/Rtmp9Jcjte/downloaded_packages/akima_0.5-4.tar.gz", repos=NULL)

>>>>>>>>>>>>>>
[root@n027 tmp]# R

R version 2.13.0 (2011-04-13)
Copyright (C) 2011 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages("/tmp/akima_0.5-4.tar.gz", repos=NULL)
* installing *source* package âakimaâ ...
** libs
gfortran   -fpic  -g -O2 -c akima.new.f -o akima.new.o
gfortran   -fpic  -g -O2 -c akima433.f -o akima433.o
gfortran   -fpic  -g -O2 -c akima697.f -o akima697.o
gfortran   -fpic  -g -O2 -c idbvip.f -o idbvip.o
gfortran   -fpic  -g -O2 -c idcldp.f -o idcldp.o
gfortran   -fpic  -g -O2 -c idgrid.f -o idgrid.o
gfortran   -fpic  -g -O2 -c idlctn.f -o idlctn.o
gfortran   -fpic  -g -O2 -c idpdrv.f -o idpdrv.o
gfortran   -fpic  -g -O2 -c idptip.f -o idptip.o
gfortran   -fpic  -g -O2 -c idptli.f -o idptli.o
gfortran   -fpic  -g -O2 -c idsfft.f -o idsfft.o
gfortran   -fpic  -g -O2 -c idtang.f -o idtang.o
gfortran   -fpic  -g -O2 -c idxchg.f -o idxchg.o
gcc -std=gnu99 -I/usr/local/src/apps/R-2.13.0/lib64/R/include  -I/usr/local/include    -fpic  -g -O2 -c init.c -o init.o
gfortran   -fpic  -g -O2 -c tripack.f -o tripack.o
gfortran   -fpic  -g -O2 -c ttidbs.f -o ttidbs.o
gcc -std=gnu99 -shared -L/usr/local/lib64 -o akima.so akima.new.o akima433.o akima697.o idbvip.o idcldp.o idgrid.o idlctn.o idpdrv.o idptip.o idptli.o idsfft.o idtang.o idxchg.o init.o tripack.o ttidbs.o -lgfortran -lm -L/usr/local/src/apps/R-2.13.0/lib64/R/lib -lR
installing to /usr/local/src/apps/R-2.13.0/lib64/R/library/akima/libs
** R
** data
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
** testing if installed package can be loaded

* DONE (akima)
>>>>>>>

Post Install

vi  /sw/R/2.13.0/lib64/R/bin/R

R_HOME_DIR=/sw/R/2.13.0/lib64/R
#R_HOME_DIR=/usr/local/src/apps/R-2.13.0/lib64/R

Additional packages

mvtnorm
> install.packages("mvtnorm", repos="http://R-Forge.R-project.org")
trying URL 'http://R-Forge.R-project.org/src/contrib/mvtnorm_0.9-9992.tar.gz'
Content type 'application/x-gzip' length 315853 bytes (308 Kb)
opened URL
==================================================
downloaded 308 Kb

* installing *source* package âmvtnormâ ...
** libs
gcc -std=gnu99 -I/sw/R/2.13.0/lib64/R/include  -I/usr/local/include    -fpic  -g -O2 -c miwa.c -o miwa.o
gfortran   -fpic  -g -O2 -c mvt.f -o mvt.o
gcc -std=gnu99 -I/sw/R/2.13.0/lib64/R/include  -I/usr/local/include    -fpic  -g -O2 -c randomF77.c -o randomF77.o
gfortran   -fpic  -g -O2 -c tvpack.f -o tvpack.o
gcc -std=gnu99 -I/sw/R/2.13.0/lib64/R/include  -I/usr/local/include    -fpic  -g -O2 -c tvpackAux.c -o tvpackAux.o
gcc -std=gnu99 -shared -L/usr/local/lib64 -o mvtnorm.so miwa.o mvt.o randomF77.o tvpack.o tvpackAux.o -lgfortran -lm -L/sw/R/2.13.0/lib64/R/lib -lR
installing to /sw/R/2.13.0/lib64/R/library/mvtnorm/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
** testing if installed package can be loaded

* DONE (mvtnorm)

The downloaded packages are in
        â/tmp/RtmpVkCqG5/downloaded_packagesâ
Updating HTML index of packages in '.Library'
Making packages.html  ... done

corpcor

R-3.2.3 Installation

module load library/atlas/3.10.2
module load jdk/1.8.0_66

./configure --prefix=/sw/R/R-3.2.3 --with-blas="-L/sw/library/atlas/3.10.2/my_build_dir/lib -lptf77blas -lpthread -latlas" --with-lapack --enable-R-shlib  --enable-BLAS-shlib LIBnn=lib64 JAVA_HOME=/sw/sdev/jdk/jdk1.8.0_66   2>&1 | tee log.configure_R

make  2>&1 | tee log.make_R
make check 2>&1 | tee log.makeCheck_R

make  install 2>&1 | tee log.make_install_R.txt
cp -rp library lib include doc m4 modules po tests tools  
cp -rp library lib include doc m4 modules po tests tools  ../R-3.2.3/
>>>>>>>>>
module display R/3.2.3
-------------------------------------------------------------------
/sw/com/modulefiles/R/3.2.3:
module-whatis     Set up environment for R version 3.2.3
conflict     R/3.0.2
conflict     R/3.2.0
conflict     R/3.2.3old
module         load jdk/1.8.0_66
module         load library/gdal/1.9.2
module         load library/proj/4.8.0
module         load library/geos/3.4.2
module         load jags/4.0.1
module         load library/atlas/3.10.2
prepend-path     PATH /sw/R/R-3.2.3/bin
prepend-path     PATH /sw/R/R-3.2.3/lib64/R/bin
prepend-path     LD_LIBRARY_PATH /sw/R/R-3.2.3/lib64/R/lib
prepend-path     INCLUDE /sw/R/R-3.2.3/lib64/R/include
prepend-path     MANPATH /sw/R/R-3.2.3/share/man
-------------------------------------------------------------------
>>>>>>>>

module load R/3.2.3
wget https://cran.r-project.org/src/contrib/pbkrtest_0.4-6.tar.gz
module load R/3.2.3
install.packages('lme4', dependencies=TRUE, repos='http://cran.rstudio.com/')
install.packages("/tmp/pbkrtest_0.4-6.tar.gz", repos=NULL)
install.packages('caret', dependencies=TRUE, repos='http://cran.rstudio.com/')

Rmpi

module load mpi/openMPI/1.8.5-gcc4.9.0
R CMD INSTALL --configure-args="--with-Rmpi-include=/sw/openMPI/1.8.5-gcc4.9.0/include --with-Rmpi-libpath=/sw/openMPI/1.8.5-gcc4.9.0/lib/openmpi --with-Rmpi-type=OPENMPI" /tmp/Rmpi_0.6-5.tar.gz

 

gpuR

module load cuda/7.5.18
install.packages('gpuR', dependencies=TRUE, repos='http://cran.rstudio.com/')

rgdal

R CMD INSTALL rgdal

 

R-3.3.3 Installation

 

Pre-Requisite

Pre-Requisite
============
>>>>>>>>>>>Install xy to take care of lzma dependency >>>>>>>>>>>>>>>>>>>>>>>>>>
xz-5.3.3
tar -zxvf xz-5.2.3.tar.gz
./configure --prefix=/sw/library/xz/5.3.3
make -j3
make install
>>>>>>>>>>>>pcre>>>>>>>>>>>>>>>
pcre-8.39
bunzip2 pcre-8.39.tar.bz2
tar -xvf pcre-8.39.tar
./configure --enable-utf8 --prefix=/sw/pcre/8.39
make -j3
make install
>>>>>>>>curl>>>
wget https://curl.haxx.se/download/curl-7.53.1.tar.gz

./configure --prefix=/sw/misc/curl/7.53.1
make -j3
make install

>>>>>>bzip2>>>>>>>>>
got error related to bizp2
“/sw/library/bzip2/1.0.6/lib/libbz2.a: could not read symbols: Bad value”

>>
>>>>>>bzip2 fix>>>>>>>>>
cd /sw/library/bzip2/src/bzip2-1.0.6fpic
vi Makefile
CFLAGS=-Wall -Winline -O2 -g $(BIGFILES) -fPIC
make  2>&1 | tee makeLog.txt
make install PREFIX=/sw/library/bzip2/1.0.6fpic  2>&1 | tee makeInstallLog.txt
>>>>>>>

Actual Installation

module purge
module load library/zlib/1.2.8
module load library/bzip2/1.0.6fpic
module load library/xz/5.3.3
module load pcre/8.39
module load misc/curl/7.53.1
module load ATLAS/3.9.39
module load jdk/1.8.0_66

./configure --prefix=/sw/R/3.3.3 '--with-cairo'  '--with-jpeglib' '--with-readline' '--with-tcltk'  '--with-blas' '--with-lapack' '--enable-R-profiling'  '--enable-R-shlib'  '--enable-memory-profiling' --enable-R-shlib  --enable-BLAS-shlib  LIBnn=lib64 JAVA_HOME=/sw/sdev/jdk/jdk1.8.0_66/jre    2>&1 | tee log.configure_R.txt

make  2>&1 | tee makeLog.txt

make install 2>&1 | tee makeInstallLog.txt

 

Ref: http://pj.freefaculty.org/blog/?p=315

R-3.4.0

module purge
module load library/zlib/1.2.8
module load library/bzip2/1.0.6fpic
module load library/xz/5.3.3
module load pcre/8.39
module load misc/curl/7.53.1
module load ATLAS/3.9.39
module load jdk/1.8.0_66
module load gcc/4.9.3
module load misc/openssl/1.0.2

 ./configure --prefix=/sw/R/3.4.0 '--with-cairo'  '--with-jpeglib' '--with-readline' '--with-tcltk'  '--with-blas' '--with-lapack' '--enable-R-profiling'  '--enable-R-shlib'  '--enable-memory-profiling' --enable-R-shlib  --enable-BLAS-shlib  LIBnn=lib64 JAVA_HOME=/sw/sdev/jdk/jdk1.8.0_66/jre  LDFLAGS=" -L/sw/pcre/8.39/lib"  CPPFLAGS="-I/sw/pcre/8.39/include"  2>&1 | tee log.configure_R.txt

make  2>&1 | tee makeLog.txt

make install 2>&1 | tee makeInstallLog.txt

 

Useful Commands to install additional packages in R

 

install.packages("foobarbaz")

ap <- available.packages()

setRepositories()

library(devtools)
install_github("packageauthor/foobarbaz")
install_bitbucket("packageauthor/foobarbaz")
install_gitorious("packageauthor/foobarbaz")

install.packages("Rbbg", repos = "http://r.findata.org")
install.packages("lubridate", dependencies=TRUE, repos='http://cran.rstudio.com/')


source("https://bioconductor.org/biocLite.R")
biocLite("preprocessCore")

install_url("http://cran.r-project.org/src/contrib/Archive/sentiment/sentiment_0.2.tar.gz")

R CMD INSTALL /tmp/RcppParallel_4.3.20.tar.gz

 

Sample PBS jobs

Serial jobs

#!/bin/bash
######################################################################
#### This section defines options for the pbs batching system
######################################################################
#PBS -m abe
#PBS -M YourEmail@griffith.edu.au
#PBS -q workq
#PBS -l select=1:ncpus=1:mem=2gb
#PBS -l walltime=60:00:00
#PBS -N vivaxIBMS3
#
###Load the pgi compilers.
module load compilers/pgi-32bit-12.4
###Load the R 
module load R/2.13.0


#####################################################################
#### This section is for my debugging purposes (not required)
######################################################################
echo Running on host `hostname`
echo Time is `date`
######################################################################
#### This section is setting up and running your executable or script
######################################################################
cd cd  $PBS_O_WORKDIR
###cd /export/home/s12345/pbs/R/
echo Directory is `pwd`

source $HOME/.bashrc
R CMD BATCH vivaxIBMS3.r resultsIBMS3.Rout
#CMD BATCH options which tells it to immediately run an R program
#instead of presenting an interactive prompt
#R.out is the screen output
#results.Rout is the R program output

More example here: http://confluence.rcs.griffith.edu.au:8080/display/v20zCluster/R#R-RunningRjobsserially

R mpi jobs

Check: http://confluence.rcs.griffith.edu.au:8080/display/GHPC/Rmpi

R And cuda:

R gputools is not currently installed as it needs EMI Photonics' CULA libraries.

http://brainarray.mbni.med.umich.edu/Brainarray/rgpgpu/ http://cran.r-project.org/web/packages/gputools/index.html

If the gputools package is installed, it will provide R interfaces to a handful of common statistical algorithms. These algorithms are implemented in parallel using a mixture of Nvidia's CUDA langauge, Nvidia's CUBLAS library, and EMI Photonics' CULA libraries. On a computer equiped with an Nvidia GPU some of these functions may be substantially more efficient than native R routines.

Sample script to run on flashlite cluster


This sample will only work on flashlite (UQ) cluster. If you are using flashlite, you may modify this template

#! /bin/bash
# #PBS -m e
#PBS -N BigMemRjob
#PBS  -l  walltime=40:00:00
#PBS -q BigMemory
###Request 1.3TB of memory
#PBS -l nodes=1:ppn=24,mem=1331200mb,vmem=1331200mb
#PBS  -A qris-gu
                             
source $HOME/.bashrc
module  load R/3.2.3
cd  /home/userNameonFlashLite/pbs
export ROutfile=ROutfile001.txt

## RUN ##
echo "=== start R-3.2.3 ==="
Rscript --no-save myRscript.R &> $ROutfile
echo "Completed\n"

 

Reference

1. http://yusung.blogspot.com.au/2009/01/install-jags-and-rjags-in-fedora.html

2. https://stackoverflow.com/questions/25721884/how-should-i-deal-with-package-xxx-is-not-available-for-r-version-x-y-z-wa

  • No labels