...
We have adopted these instructions from Princeton Research Computing HPC wiki page. We acknowledge it as the source and the page to go to for running jupyter notebooks on HPC.
Installation
Please use one of the approaches described on this page to carry out your work
Using Conda Environments: create a Conda environment on the login node
...
Internet is Not Available on Compute Nodes. Jupyter sessions will have to run on the compute nodes which do not have Internet access. This means that you will not be able to download files, clone a repo from GitHub, install packages, etc. You will need to perform these operations on the login node before starting the session. You can run commands which need Internet access on the login nodes (gc-prd-hpclogin1). Any files that you download while on the login node will be available on the compute nodes.
| No Format |
|---|
# from behind VPN if off-campus or on wireless
ssh |
...
snumber@gc-prd-hpclogin1.rcs.griffith.edu.au module load anaconda3/2020.11 |
...
source activate snumber-tf-cpu #e.g source activate s123456-tf-cpu jupyter-notebook --no-browser --port=8889 --ip=127.0.0.1 # note the last line of the output which will be something like http://127.0.0.1:8889/?token=61f8a2aa8ad5e469d14d6a1f59baac05a8d9577916bd7eb0 # leave the session running |
Then
...
in
...
a
...
new
...
terminal
...
on
...
your
...
laptop,
...
| No Format |
|---|
ssh -N -f -L localhost:8889:localhost:8889 snumber@n059.rcs.griffith.edu.au |
...
...
Choose "New" then "Python 3" to launch a new notebook. Note that Jupyter may use a port that is different than the one you specified. This is why it is import to copy and paste the URL.
When you are done, terminate the ssh tunnel by running lsof -i tcp:8889 to get the PID and then kill -9 <PID> (e.g., kill -9 6010).
Custom Conda Environment
The procedue above will only be useful if you only need the base Conda environment which includes just less than three hundred packages. If you need custom packages then you should create a new Conda environment and include jupyter in addition to the other packages that you need. The necessary modifications are shown below:
...
Internet access is available when running Jupyter on a OnDemand node. There is no job scheduler on the onDemand nodes. Be sure to use these nodes in a way that is to fair all users.
| No Format |
|---|
# from behind VPN if off-campus or on wireless
ssh snumber@n059,rcs,griffith.edu.au
module load anaconda3/2020.11
source activate snumber-tf-cpu #e.g source activate s123456-tf-cpu
jupyter-notebook --no-browser --port=8889 --ip=127.0.0.1
# note the last line of the output which will be something like
http://127.0.0.1:8889/?token=61f8a2aa8ad5e469d14d6a1f59baac05a8d9577916bd7eb0
# leave the session running |
Then in a new terminal on your laptop,
| No Format |
|---|
ssh -N -f -L localhost:8889:localhost:8889 snumber@n059.rcs.griffith.edu.au |
Lastly, open a web browser and copy and paste the URL from the previous output:
http://127.0.0.1:8889/?token=61f8a2aa8ad5e469d14d6a1f59baac05a8d9577916bd7eb0
Choose "New" then "Python 3" to launch a new notebook. Note that Jupyter may use a port that is different than the one you specified. This is why it is import to copy and paste the URL.
When you are done, terminate the ssh tunnel by running lsof -i tcp:8889 to get the PID and then kill -9 <PID> (e.g., kill -9 6010).
Reference
1. https://jupyter.org/try
2. https://researchcomputing.princeton.edu/support/knowledge-base/jupyter