Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

No Format
The usage is:
module load anaconda3/2019.07.py3
OR
module load anaconda/4.00py2.7
OR:
module load anaconda/4.00py3.5
 
You need to run this after loading the modules:
source /sw/anaconda/py27-ver400/etc/profile.d/conda.sh  #if using module load anaconda/4.00py2.7
OR source /sw/anaconda/py3-ver400/etc/profile.d/conda.sh #if using “module load anaconda/4.00py3.5”

 Manual Usage:
You will need to activate the environment. For example, to activate biopython

source activate biopython
discarding /sw/anaconda/2.0.1/bin from PATH
prepending /sw/anaconda/2.0.1/envs/biopython/bin to PATH

To deactivate this environment, use:
source deactivate

Module Usage:
=============
For example,You can use this to load biopython using python 2.7 version (anaconda) :

module load anaconda/py27biopython1.66


For python 3.5 (anaconda env), use this:
module load anaconda/py35biopython1.66


Anaconda environment need not be loaded separately.
 
However, it can be loaded as follows if needed for other reasons:

module load anaconda/4.00py2.7

module load anaconda/4.00py3.5 




...

Installed Environment

 
No Format

 


Useful commands

No Format
Useful commands:
 
conda info --envs
conda list
conda activate

...

No Format
#!/bin/bash
#PBS -N condatest
#PBS -m abe
#PBS -M YourEMAIL@griffith.edu.au
#PBS -l select=1:ncpus=1:mem=4gb,walltime=10:00:00
module load anaconda/4.00py2.7
source /sw/anaconda/py27-ver400/etc/profile.d/conda.sh 
##use /sw/anaconda/py3-ver400/etc/profile.d/conda.sh if using “module load anaconda/4.00py3.5”
echo "Starting job: "
cd  $PBS_O_WORKDIR
python <pythonFile>
 
>>>An example of the python file>>>>>>>>> 
cat pythonFile.py
import tensorflow as tf
>>>>>>>>>>>>>> 

 


Activate an environment

No Format
source /sw/anaconda/py27-ver400/etc/profile.d/conda.sh
OR source /sw/anaconda/py3-ver400/etc/profile.d/conda.sh if using “module load module load anaconda/4.00py3.5”
source activate biopython

e.g:

source activate biopython
discarding /sw/anaconda/2.0.1/bin from PATH
prepending /sw/anaconda/2.0.1/envs/biopython/bin to PATH

...

No Format
conda remove -n biopython


n060 anaconda installation

Reference

...

No Format
module load anaconda/5.3.1py3

conda info --envs
# conda environments:
#
base                  *  /sw/centos7/anaconda/5.3.1/py3
accelerate               /sw/centos7/anaconda/5.3.1/py3/envs/accelerate
biopython                /sw/centos7/anaconda/5.3.1/py3/envs/biopython
deeplearning             /sw/centos7/anaconda/5.3.1/py3/envs/deeplearning
keras                    /sw/centos7/anaconda/5.3.1/py3/envs/keras
tensorflow-gpu           /sw/centos7/anaconda/5.3.1/py3/envs/tensorflow-gpu

e.g
module load anaconda/5.3.1py3
source activate keras
pip install soundfile

Installing packages without root access

No Format
Instructions from: https://medium.com/@danielwzou/installing-spacy-and-other-packages-without-sudo-or-root-privileges-using-anaconda-59571ec0ecad

First, navigate to your home directory.
module load anaconda/5.3.1py3
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh
Now, we have a local installation of the Conda package manager at ~/miniconda3
Let’s add conda to our bash profile so that we can call it from the command line
vim .bash_profile
i
Add this line to the file:
export PATH=$HOME/miniconda3/bin:$PATH
esc :wq
Now, let’s reload our .bash_profile
source .bash_profile
And check that our installation was completed successfully
which conda
Which should return ~/miniconda3/bin/conda
Now that Conda is installed, we can install Spacy or other Python packages
conda install spacy
python -m spacy download en
Now, let’s make sure that Spacy is installed correctly
python
import spacy
nlp = spacy.load('en')
doc = nlp(u'Hello World!')
print([w.lemma_ for w in doc])
Now that you have Conda installed, you can use it to install many packages locally.


Anaconda on the new cluster

No Format
module load anaconda3/2019.07py3
 
This should have packages like the tensorflow env. You can find with this:
conda info --envs
 
>>>>> 
conda info --envs
# conda environments:
#
                         /sw/anaconda2/2019.07/envs/bio
                         /sw/anaconda2/2019.07/envs/bioinfoPy27
base                  *  /sw/anaconda3/2019.07
3point6                  /sw/anaconda3/2019.07/envs/3point6
R                        /sw/anaconda3/2019.07/envs/R
bioinformatics           /sw/anaconda3/2019.07/envs/bioinformatics
 
>>>>>>> 
This will give the packages in the base:
conda list
 
source activate 3point6
Next run: " conda list"
This will list the packages installed in that environment.
 
It shows among many others:
pytorch                   1.3.1           cuda100py36h53c1284_0
 
tensorboard               2.0.0              pyhb230dea_0
tensorflow                2.0.0           mkl_py36hef7ec59_0
tensorflow-base           2.0.0           mkl_py36h9204916_0
tensorflow-estimator      2.0.0              pyh2649769_0
termcolor                 1.1.0                      py_2    conda-forge
theano                    1.0.4           py36hf484d3e_1000    conda-forge
 
Remember this cluster has no GPU (currently)


Another sample pbs file


No Format
#!/bin/bash
#PBS -N condatest
#PBS -m abe
#PBS -M YourEMAIL@griffith.edu.au
#PBS -l select=1:ncpus=1:mem=4gb,walltime=10:00:00
cd  $PBS_O_WORKDIR
module load anaconda3/2019.07py3 
source activate bioinformatics
trinity <trinityinput>



Sample python code to write out a plot


Code Block
vi plot.py
>>>>>>>>>>>>>>>>>
import matplotlib.pyplot as plt
import numpy as np

np.random.seed(19680801)
data = np.random.randn(2, 100)

fig, axs = plt.subplots(2, 2, figsize=(5, 5))
axs[0, 0].hist(data[0])
axs[1, 0].scatter(data[0], data[1])
axs[0, 1].plot(data[0], data[1])
axs[1, 1].hist2d(data[0], data[1])

plt.show()
plt.savefig('test')
>>>>>>>>>>>>>>>>>>

module load anaconda3/2022.10
source activate yourenv #this should have the matlibplot packages and others.
python plot.py  would generate a plot named test.png

Reference: https://matplotlib.org/3.4.3/tutorials/introductory/sample_plots.html
Sample codes are here:
https://matplotlib.org/3.4.3/_downloads/55a714c8a6e4dde456b09326202fe492/sample_plots.py

https://matplotlib.org/3.4.3/_downloads/003118d8eb1e7df55671bcdf18bfac3c/sample_plots.ipynb



Ref: https://medium.com/@danielwzou/installing-spacy-and-other-packages-without-sudo-or-root-privileges-using-anaconda-59571ec0ecad

Reference

1. http://davebehnke.com/using-python-anaconda-distribution.html
2. https://store.continuum.io/cshop/anaconda/

3. https://medium.com/@danielwzou/installing-spacy-and-other-packages-without-sudo-or-root-privileges-using-anaconda-59571ec0ecad
4.
http://continuum.io/downloads#all